System

Abstract

A system is provided.SOLUTION: A system comprising: means for collecting personal information provided by a user; means for integrating and collecting information from a plurality of account services; means for pre-processing and inputting the collected data into a learning model; means for training the machine learning model using the pre-processed data; means for generating an optimal answer to a question from the user; and means for providing the generated answer to the user.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The technology of the present disclosure relates to a system. [Background technology]

[0002] Patent document 1 discloses a persona chatbot control method performed by at least one processor, the method including the steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to a description of the chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2022-180282 Summary of the Invention [Problem to be solved by the invention]

[0004] Current generative AI systems require users to input detailed and specific questions, which is time-consuming. Furthermore, the answers generated are often general and do not address individual user needs. As a result, many users are unable to fully enjoy the convenience of AI systems. The objective of the present invention is to provide a system that makes maximum use of a user's individual information and provides optimal answers simply by asking a single question. [Means for solving the problem]

[0005] The present invention solves the above problems by providing the following means: A system is provided that includes a means for collecting personal information provided by a user, a means for integrating and collecting information from multiple account services, a means for preprocessing the collected data and inputting it into a learning model, a means for training a machine learning model using the preprocessed data, a means for generating optimal answers to questions from the user, and a means for providing the generated answers to the user. This allows a user to obtain an optimal answer tailored to their individual needs by simply asking a single question.

[0006] "User Information" means any personal data or information provided by a User.

[0007] "Account information" refers to authentication and usage information that a user uses across multiple services.

[0008] "Collection methods" refer to the methods and tools used to collect information from users and each service.

[0009] "Preprocessing" refers to a series of operations that transform collected data into a format suitable for machine learning models.

[0010] A "learning model" is an artificial intelligence structure or algorithm that learns patterns and rules using collected and preprocessed data.

[0011] An "optimal solution" is an answer or suggestion that best suits the user's needs and situation.

[0012] "Provision means" refers to the methods and tools used to communicate the generated optimal solution to the user.

[0013] "Filtering" refers to the process of removing unnecessary information.

[0014] "Normalization" is the operation of converting data into a uniform form or scale. [Brief explanation of the drawings]

[0015] [Figure 1] 1 is a conceptual diagram showing an example of the configuration of a data processing system according to a first embodiment. [Figure 2] 1 is a conceptual diagram showing an example of main functions of a data processing device and a smart device according to a first embodiment. [Figure 3] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a second embodiment. [Figure 4] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and smart glasses according to a second embodiment. [Figure 5] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a third embodiment. [Figure 6] FIG. 11 is a conceptual diagram showing an example of main functions of a data processing device and a headset-type terminal according to a third embodiment. [Figure 7] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a fourth embodiment. [Figure 8] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and a robot according to a fourth embodiment. [Figure 9] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 10] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 11] FIG. 3 is a sequence diagram showing a processing flow of the data processing system according to the first embodiment. [Figure 12] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 1. [Figure 13] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system according to the second embodiment when an emotion engine is combined. [Figure 14] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 2 when an emotion engine is combined. DETAILED DESCRIPTION OF THE INVENTION

[0016] An example of an embodiment of a system according to the technology of the present disclosure will be described below with reference to the accompanying drawings.

[0017] First, the terms used in the following description will be explained.

[0018] In the following embodiments, a coded processor (hereinafter simply referred to as a "processor") may be a single arithmetic device or a combination of multiple arithmetic devices. Furthermore, a processor may be a single type of arithmetic device or a combination of multiple types of arithmetic devices. Examples of arithmetic devices include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), and an APU (Accelerated Processing Unit).

[0019] In the following embodiments, a coded RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a working memory by a processor.

[0020] In the following embodiments, the coded storage is one or more non-volatile storage devices that store various programs, various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), and magnetic tapes.

[0021] In the following embodiments, a communication I / F (Interface) with a symbol is an interface including a communication processor, an antenna, etc. The communication I / F controls communication between multiple computers. Examples of communication standards applied to the communication I / F include wireless communication standards including 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), Bluetooth (registered trademark), etc.

[0022] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." In other words, "A and / or B" means that it may be only A, only B, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" is also applied when three or more things are expressed connected by "and / or."

[0023] [First embodiment]

[0024] FIG. 1 shows an example of the configuration of a data processing system 10 according to the first embodiment.

[0025] 1, a data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0026] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0027] The smart device 14 includes a computer 36, a reception device 38, an output device 40, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The reception device 38, the output device 40, and the camera 42 are also connected to the bus 52.

[0028] The reception device 38 includes a touch panel 38A, a microphone 38B, and the like, and receives user input. The touch panel 38A detects contact with an indicator (for example, a pen or a finger) to receive user input by the touch of the indicator. The microphone 38B detects the user's voice to receive user input by voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0029] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form of expression that the user 20 can perceive (for example, audio and / or text). The display 40A displays visible information such as text and images in accordance with instructions from the processor 46. The speaker 40B outputs audio in accordance with instructions from the processor 46. The camera 42 is a compact digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0030] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54.

[0031] FIG. 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0032] 2, in the data processing device 12, a specific process is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific process is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0033] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0034] In the smart device 14, the processor 46 performs the reception output process. The storage 50 stores a reception output program 60. The reception output program 60 is used in conjunction with the specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0035] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0036] The present invention is a system that collects and integrates personal information and information from multiple account services provided by users, inputs the preprocessed data into a machine learning model, and trains it to provide optimal answers to questions from users.

[0037] System configuration and operation

[0038] Data Collection Phase

[0039] The server first collects personal information provided by the user, including name, age, address, income, etc. The terminal prompts the user to enter this information and sends it to the server.

[0040] The server then aggregates and collects information from multiple account services (e.g., communication services, messaging services, search services, etc.). To obtain this information, it uses the APIs of each service to obtain the necessary user data. The device provides an interface for the user to enter their API key and authentication information.

[0041] Data preprocessing phase

[0042] The server preprocesses the collected data and converts it into a format suitable for the learning model. Preprocessing includes filtering unnecessary information and normalizing the data. This procedure improves the quality of the data and optimizes the performance of the learning model.

[0043] Learning Phase

[0044] The server uses the preprocessed data to train a machine learning model, which then creates a model that reflects the individual needs and behavioral patterns of the user. The server then stores the trained model and uses it for subsequent query processing.

[0045] Inquiry Processing Phase

[0046] When a user types a one-word question into their device, the device sends the question to the server, which uses the trained model to generate the best answer for the question. The generated answer is customized based on the individual user profile, so it can meet the user's needs with high accuracy.

[0047] Finally, the terminal displays the answer received from the server to the user, and a special user interface allows the user to intuitively understand the answer and ask for more detailed information if necessary.

[0048] Specific examples

[0049] For example, consider a case where a user is looking for a new refrigerator. The user enters a one-word question such as, "I want a new refrigerator." When the device sends this question to the server, the server will suggest the optimal refrigerator model based on the user's financial situation and living environment.

[0050] The server selects the refrigerator with the best price and energy efficiency, taking into account the user's income and the electricity rates in the area where they live, for example. It also provides information on local recycling services and discount campaigns. In this way, users can obtain information on the refrigerator that is best suited to them by simply asking a question.

[0051] The above is an embodiment of the present invention, and the specific configuration and operation of the system have been described. This system allows users to easily obtain optimal information, thereby providing high convenience.

[0052] The processing flow will be explained below.

[0053] Step 1:

[0054] The user enters personal information, including name, age, address, income, etc. The entered information is collected by the terminal and sent to the server.

[0055] Step 2:

[0056] The server aggregates and collects information from multiple account services (e.g., communication services, messaging services, search services). The device provides an interface for users to enter API keys and authentication information. The server obtains user data using the APIs of each service.

[0057] Step 3:

[0058] The server preprocesses the collected personal and account information, which includes filtering out unnecessary information and normalizing the data, converting it into a format suitable for the learning model.

[0059] Step 4:

[0060] The server uses the preprocessed data to train a machine learning model. This training process creates a model that reflects the individual needs and behavioral patterns of the user. Once trained, the model is stored on the server.

[0061] Step 5:

[0062] The user inputs a one-word question into the terminal, for example, "I want a new refrigerator." The terminal then sends this question to the server.

[0063] Step 6:

[0064] The server uses the trained model to generate optimal answers to user questions, and suggests optimal products and services based on the user's profile and the content of the question.

[0065] Step 7:

[0066] The terminal displays the answer received from the server to the user, which may include, for example, recommended refrigerator models and prices, local recycling services, and discount campaign information.

[0067] The above is the specific processing flow of the system program. This allows users to easily obtain the most appropriate information, providing a high level of convenience.

[0068] Example 1

[0069] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0070] In conventional systems, when integrating and using information provided by users with information from multiple account services, it has been difficult to ensure data accuracy and consistency, and to provide optimal information to individual users. Furthermore, there has been a need to efficiently train machine learning models and generate fast, highly accurate answers to user questions. The present invention aims to solve these problems and provide users with highly accurate, optimal information.

[0071] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0072] In this invention, the server includes: means for collecting personal information provided by a user; means for integrating and collecting information from multiple account services; means for preprocessing the collected data and inputting it into a learning model; means for training a machine learning model using the preprocessed data; means for generating optimal answers to questions from users; means for a terminal to acquire authentication information from the user and send it to the server; means for the server to acquire user data using APIs of each service; means for the server to normalize data and filter unnecessary data; means for training the preprocessed data using a machine learning algorithm and saving the model; and means for accepting user questions, generating answers using the saved model, and sending them to the terminal. This makes it possible to provide users with quick and accurate answers that meet their individual needs.

[0073] A "user" is an individual or group that uses the system and provides personal information, authentication information, etc.

[0074] A "server" is a computer system that collects, stores, and processes information provided by users and trains and executes machine learning models.

[0075] A "terminal" is a device that a user uses to enter information or submit a question, and is an interface that communicates with a server.

[0076] "Personal information" refers to basic information about a user, including data such as name, age, address, and income.

[0077] "Account services" refer to various online services used by users, including communication services, messaging services, search services, and the like.

[0078] "API" stands for Application Program Interface, a set of rules and protocols for exchanging data between different software programs.

[0079] "Data normalization" is the process of converting collected data into a consistent format, a process undertaken to improve data quality.

[0080] "Data filtering" is the process of removing information from a dataset that is not useful to a machine learning model.

[0081] A "machine learning model" is an algorithm or system that uses collected and preprocessed data to learn and perform a specific task.

[0082] A "learning algorithm" is a mathematical or statistical method for using data to train a machine learning model and improve its performance.

[0083] "Authentication information" refers to information required for a user to access each account service, including API keys and passwords.

[0084] A "question" is an inquiry that a user inputs into the system, and is information that triggers the server to generate the most appropriate answer.

[0085] An "answer" is a response generated by the server to a user's question, customized based on the user's profile and data.

[0086] MODE FOR CARRYING OUT THE INVENTION

[0087] The present invention is a system that collects and integrates personal information and information from multiple account services provided by users, inputs the preprocessed data into a machine learning model, and trains it to provide optimal answers to questions from users.

[0088] System Configuration

[0089] The system mainly consists of a server, a terminal, and a user. Each component is explained in detail below.

[0090] Data Collection Phase

[0091] The user uses the device to enter personal information, such as name, age, address, and income. The entered information is sent to the server via the device. The server then retrieves the user data using APIs of multiple account services (e.g., communication services, messaging services, search services, etc.). The device provides an interface for the user to enter the necessary API keys and authentication information. Protocols such as REST API and OAuth are used to collect this type of data.

[0092] Data preprocessing phase

[0093] The server preprocesses the collected data, converting it into a format suitable for the learning model. Preprocessing includes filtering out unnecessary information and normalizing the data. For example, it converts address information into a standard format and standardizes the currency unit of income information. Techniques used include regular expressions and statistical methods.

[0094] Learning Phase

[0095] The server uses the preprocessed data to train a machine learning model. For training, a machine learning framework such as TensorFlow or PyTorch is used. This creates a model that reflects the individual needs and behavioral patterns of the user. The trained model is saved in the server's file system or database. Common save formats include model.h5 and model.pt.

[0096] Inquiry Processing Phase

[0097] When a user inputs a question from their device, the device sends the question to the server, which uses the trained model to generate the optimal answer to the question. For example, if a user asks, "I want a new refrigerator," the server will suggest the optimal refrigerator model based on the user's financial situation and living environment. The answer is sent to the device and displayed to the user.

[0098] Specific examples

[0099] For example, consider a case where a user inputs "I want a new refrigerator." When the device sends this question to the server, the server will suggest the optimal refrigerator model based on the user's personal information and collected account service data. Taking into account income and local electricity rates, the server will select the refrigerator with the best price range and energy efficiency, and also provide information on local recycling services and discount campaigns.

[0100] Here is an example prompt:

[0101] User: I want a new refrigerator.

[0102] Terminal: Sends user questions to the server.

[0103] Server: Recommends the optimal refrigerator model based on the user's financial situation and living environment. For example, it selects the refrigerator with the best price range and energy efficiency, taking into account income and local electricity rates, and also provides information on local recycling services and discount campaigns.

[0104] Terminal: Shows the suggested refrigerator information to the user.

[0105] This system allows users to easily obtain the most appropriate information, providing a high level of convenience.

[0106] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0107] System processing steps

[0108] Step 1: Collect user information

[0109] Input: The user enters personal information into the device.

[0110] Specific behavior: The user adds personal information such as name, age, address, income, etc. to the input form. Once the input is complete, the user clicks the "Submit" button.

[0111] Data processing: The terminal sends the entered personal information to the server via the REST API.

[0112] Output: The server receives the user's personal information and stores it in a database.

[0113] Step 2: Integrate your account service information

[0114] Input: The user enters the authentication information for each account service into the device.

[0115] Specific operation: The user enters the API key and authentication information into the input form and clicks the "Submit" button.

[0116] Data processing: The terminal sends authentication information to the server via REST API, and the server retrieves user data using the API of each service.

[0117] Output: The server retrieves user data from each account service and integrates it into a database.

[0118] Step 3: Normalize the data

[0119] Input: Personal information and account service information collected by the server

[0120] Specific operation: The server converts the address information into a unified format and unifies the currency unit of the income information.

[0121] Data manipulation: Using regular expressions and statistical techniques to transform data into a consistent format.

[0122] Output: The server generates the normalized data and stores it in a database.

[0123] Step 4: Filtering out unnecessary data

[0124] Input: Server-normalized data

[0125] Specific operation: The server filters out unnecessary information (e.g., phone numbers, email addresses).

[0126] Data processing: The server uses scripts to automatically remove unnecessary fields.

[0127] Output: The server generates the filtered data and stores it in a database.

[0128] Step 5: Prepare the training data

[0129] Input: Server filtered data

[0130] What it does: The server converts the data into a format suitable for the machine learning model.

[0131] Data processing: The preprocessed data is formatted and converted into a format that is easy for machine learning algorithms to use (e.g., CSV or JSON).

[0132] Output: The server generates a dataset that can be trained on.

[0133] Step 6: Training the model

[0134] Input: Training data prepared by the server

[0135] Specific operation: The server trains machine learning models using TensorFlow or PyTorch.

[0136] Data processing: The server inputs the training data into the algorithm to train the model.

[0137] Output: The server generates the trained model and saves it to the file system (in the format model.h5 or model.pt).

[0138] Step 7: Ask questions

[0139] Input: The user types a question into the terminal

[0140] Specific behavior: The user enters a question into the input form and clicks the "Submit" button.

[0141] Data processing: The terminal sends the question to the server via the REST API.

[0142] Output: The server receives the query and records it in a database for processing.

[0143] Step 8: Generate an answer

[0144] Input: The question received by the server and the trained model

[0145] What it does: The server uses the trained model to analyze the question and generate the best answer.

[0146] Data Calculation: The server analyzes the question and generates an answer based on the user profile and model.

[0147] Output: The server records the generated answer in a database and sends it to the terminal.

[0148] Step 9: View your answers

[0149] Input: The answer sent by the server

[0150] Specific operation: The terminal displays the answer received from the server to the user.

[0151] Data processing: The terminal embeds the response data into a template and converts it into a format for display.

[0152] Output: The user can check the answer on the device.

[0153] The above are the specific processing steps of this system, which allows users to easily obtain optimal information quickly and accurately.

[0154] (Application example 1)

[0155] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0156] Today, many users use multiple electronic payment services, and checking the benefits and campaign information for each service individually is a significant burden. It is also difficult to provide accurate savings advice based on the user's consumption patterns. As a result, many users miss out on appropriate savings methods and useful campaign information. Therefore, there is a need for a system that can integrate and collect information from users' personal information and multiple account services, and provide optimal savings advice and campaign information based on each user's individual consumption patterns.

[0157] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0158] In this invention, the server includes means for collecting personal information provided by a user, means for integrating and collecting information from multiple account services, means for preprocessing the collected data and inputting it into a learning model, means for training a machine learning model using the preprocessed data, means for analyzing the user's consumption pattern and generating optimal saving methods and campaign information, and means for providing the generated saving methods and campaign information to the user. This enables the user to integrate and analyze data from multiple information sources and easily obtain accurate saving advice and useful campaign information based on their own consumption pattern.

[0159] "User" refers to an individual or corporation that uses the system.

[0160] "Personal Information" refers to information that can be used to identify a specific individual, such as a user's name, age, address, or income.

[0161] "Account Service" refers to a service that provides information related to accounts created by users with electronic payment services and other online services.

[0162] "Means for integrating and collecting information" refers to the process of centralizing and collecting user data from multiple account services.

[0163] "Preprocessing" refers to the process of filtering unnecessary information from collected data and normalizing the data.

[0164] "Means for inputting to a learning model" refers to a method for feeding preprocessed data to a machine learning algorithm for learning.

[0165] A "machine learning model" refers to an algorithm that uses large amounts of data to learn specific patterns and relationships and then makes predictions and classifications for new data.

[0166] "Consumption patterns" refers to information obtained by identifying specific spending tendencies or behavioral patterns from a user's past spending history.

[0167] "Savings tips" refer to specific measures or techniques suggested to help users reduce their spending.

[0168] "Campaign Information" refers to information regarding discounts and benefits offered by electronic payment services and related services.

[0169] "Means of generation" refers to the process of using a machine learning model to create optimal savings methods and campaign information based on a user's individual information.

[0170] The "means of providing" refers to a method of communicating the generated savings methods and campaign information to users.

[0171] The present invention is a system that integrates personal information collected from users and information from multiple account services, and uses the preprocessed data to train a machine learning model. The method by which this system provides optimal answers to user questions is described in detail below.

[0172] System configuration and operation

[0173] Data Collection Phase

[0174] The server first collects personal information provided by the user, including name, age, address, income, etc. The terminal prompts the user to enter this information and sends it to the server.

[0175] The server then aggregates and collects information from multiple account services (such as electronic payment services and other online services). To obtain this information, it uses the API of each service to obtain the necessary user data. The device provides an interface for the user to enter their API key and authentication information.

[0176] Data preprocessing phase

[0177] The server preprocesses the collected data and converts it into a format suitable for the learning model. Preprocessing includes filtering unnecessary information and normalizing the data. This procedure improves the quality of the data and optimizes the performance of the learning model.

[0178] Learning Phase

[0179] The server uses the preprocessed data to train a machine learning model, which then creates a model that reflects the individual user's consumption patterns. The server then stores the trained model and uses it for subsequent query processing.

[0180] Inquiry Processing Phase

[0181] When a user inputs a question about saving money or campaign information on their device, the device sends the question to the server, which uses the trained model to generate the optimal answer to the question. The generated answer is customized based on the individual user profile, so it can meet the user's needs with high accuracy.

[0182] Finally, the terminal displays the answer received from the server to the user, and a special user interface allows the user to intuitively understand the answer and ask for more detailed information if necessary.

[0183] Specific examples

[0184] For example, consider a case where a user wants to know about their future spending patterns. The user inputs the question, "Please tell me about your future spending patterns." When the device sends this question to the server, the server will suggest optimal savings methods and useful campaign information based on the user's income and past spending data.

[0185] Specific prompt examples:

[0186] Prompt: Tell us about your future spending patterns

[0187] Response generation context:

[0188] Username: Taro

[0189] Age: 30

[0190] Income: 500,000 yen per month

[0191] Historical spending data: [Provided in list format]

[0192] Generated response:

[0193] Basic money saving advice

[0194] Electronic payment service benefits and campaign information

[0195] In this way, the system of the present invention provides useful information to the user and achieves high convenience.

[0196] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0197] Step 1:

[0198] Data collection

[0199] The server first collects personal information (such as name, age, address, and income) from the user via the device. This input data is sent to the server in JSON format.

[0200] Next, the server obtains the user's transaction history and account information through the APIs of multiple electronic payment services, and the user enters the API key and authentication information on the terminal.

[0201] The server integrates the acquired data and stores it as a single dataset. The input is user data from each service, and the output is the integrated dataset.

[0202] Step 2:

[0203] Data Preprocessing

[0204] The server then filters unnecessary information from the combined data set and extracts only the information needed, such as past transaction history and income data.

[0205] The extracted data is then normalized, which improves the quality of the data and allows the learning model to analyze it more efficiently.

[0206] The preprocessed data is converted into a format suitable for the machine learning model. The input is the unfiltered dataset, and the output is the preprocessed dataset.

[0207] Step 3:

[0208] Training a machine learning model

[0209] The server uses the preprocessed data to train a machine learning model, using algorithms such as Linear Regression and Random Forest.

[0210] The training process creates a model that reflects your income and spending patterns. The input is the preprocessed dataset, and the output is the trained model.

[0211] The trained model is stored on the server and used for subsequent query processing.

[0212] Step 4:

[0213] Query Processing

[0214] The user inputs a question into the terminal. For example, a simple question such as "Please tell me about your future spending patterns" is input.

[0215] The terminal sends a question from the user to the server.

[0216] Step 5:

[0217] Generating optimal answers

[0218] The server uses the trained model to generate the best answer to the user's question. Based on this prompt, the model generates an answer based on the user's consumption patterns.

[0219] The generated answers are customized for each user and include specific savings methods and campaign information. The input is the user's question and the trained model, and the output is the optimal answer.

[0220] Step 6:

[0221] Providing answers

[0222] The server sends the generated response to the terminal.

[0223] The terminal displays the answer received from the server to the user, who can then consider specific actions based on the presented information. The input is the optimal answer, and the output is the displayed information.

[0224] Furthermore, an emotion engine that estimates the user's emotion may be combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59 and perform identification processing using the user's emotion.

[0225] The present invention is a system that collects and integrates personal information and information from multiple account services provided by users, inputs the preprocessed data into a machine learning model for learning, and further recognizes the user's emotions and appropriately uses the results to provide optimal answers to questions from users.

[0226] System configuration and operation

[0227] Data Collection Phase

[0228] The server first collects personal information provided by the user, such as name, age, address, and income, and the terminal prompts the user to enter this information and sends it to the server.

[0229] The server then aggregates and collects information from multiple account services (e.g., communication services, messaging services, search services, etc.). To obtain this information, it uses the APIs of each service to obtain the necessary user data. The device provides an interface for the user to enter their API key and authentication information.

[0230] Data preprocessing phase

[0231] The server pre-processes the collected personal and account information, which includes filtering unnecessary information and normalizing the data. This procedure improves the quality of the data and optimizes the performance of the learning model.

[0232] Learning Phase

[0233] The server uses the preprocessed data to train a machine learning model. This training process creates a model that reflects the individual needs and behavioral patterns of the user. Once the model is trained, it is stored on the server.

[0234] Emotion Recognition Phase

[0235] The server is equipped with an emotion engine that recognizes the user's emotions. When the user inputs a question, the device sends the input voice or text to the server, which analyzes it and recognizes the user's emotions. The emotion engine analyzes the nuances of the text and the tone of the voice to determine whether the user is excited, calm, or stressed.

[0236] Inquiry Processing Phase

[0237] When a user inputs a one-word question from their device, the device sends the question to the server, which uses the trained model to generate an optimal answer to the question. The generated answer is adjusted based on the user's individual profile as well as the user's emotions recognized by the emotion engine. This adjustment provides an answer that is most appropriate for the user's current mental state.

[0238] Finally, the terminal displays the answer received from the server to the user, and a special user interface allows the user to intuitively understand the answer and ask for more detailed information if necessary.

[0239] Specific examples

[0240] For example, imagine a user is looking for a new refrigerator. The user enters a one-word question: "I want a new refrigerator." If the emotion engine detects that the user is a little excited, the server generates a response in a tone appropriate to that emotion. For example, the response might be something like, "Thank you for your interest. The perfect refrigerator for you is the YY model from XX. Also, please note that there is a discount available at your local recycling center." This provides clear and helpful information while moderating the user's excitement.

[0241] In this way, by combining emotion recognition functionality, the present invention makes it possible to provide information tailored to individual user needs with even greater precision. This allows users to obtain the information most suited to them without stress, simply by asking a simple question, and enjoys high convenience.

[0242] The processing flow will be explained below.

[0243] Step 1:

[0244] The user enters personal information, including name, age, address, income, etc. The entered information is collected by the terminal and sent to the server.

[0245] Step 2:

[0246] The server collects and integrates information from multiple account services (e.g., communication services, messaging services, search services). The device provides an interface for the user to enter API keys and authentication information, and the server obtains user data using the APIs of each service.

[0247] Step 3:

[0248] The server preprocesses the collected personal and account information. This preprocessing includes filtering out unnecessary information and normalizing the data. This preprocessing improves the quality of the data and converts it into a format suitable for the learning model.

[0249] Step 4:

[0250] The server uses the preprocessed data to train a machine learning model. This training process creates a model that reflects the individual needs and behavioral patterns of the user. Once trained, the model is stored on the server.

[0251] Step 5:

[0252] The user inputs a question into the terminal. For example, the user inputs a question such as "I want a new refrigerator." The terminal then transmits the input text or voice data to the server.

[0253] Step 6:

[0254] The server uses an emotion engine to analyze the user's input text and voice to recognize the user's emotions. The emotion engine analyzes, for example, the nuances of the text and the tone of the voice to determine whether the user is excited, calm, or stressed.

[0255] Step 7:

[0256] The server uses the trained model to generate the best answer to the user's question, and the emotion engine adjusts the answer based on the user's emotions. For example, if the user is excited, the server creates an answer with a calming tone.

[0257] Step 8:

[0258] The server sends the generated answer to the terminal, which displays the answer to the user. Specifically, the answer includes information about the suggested products and special offers.

[0259] Step 9:

[0260] The user checks the answers displayed on the device and requests more detailed information if necessary. This process allows the user to obtain the information they need quickly and accurately without stress.

[0261] The above is the specific processing flow of the system. With this system, users can ask a simple question and get a highly accurate answer that is best suited to their individual needs.

[0262] Example 2

[0263] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0264] Conventional user support systems struggle to generate appropriate answers based on the user's individual needs. Furthermore, they are unable to generate answers that take the user's emotional state into account, which can result in a poor user experience. Furthermore, insufficient preprocessing of collected information results in suboptimal performance of machine learning models. Therefore, there is a need for a system that can provide emotionally sensitive answers while addressing the user's individual needs.

[0265] The identification process by the identification processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means. In this invention, the server includes a means for recognizing a user's emotion, a means for adjusting a response based on the user's emotion, a means for collecting personal information provided by the user, a means for integrating and collecting information from multiple account services, a means for preprocessing the collected data and inputting it into a machine learning model, a means for training the machine learning model using the preprocessed data, a means for generating an optimal answer to a question from the user, and a means for providing the generated answer to the user. This makes it possible to provide an optimal answer that takes into account the individual needs and emotional state of the user.

[0266] A "user" is an entity that inputs personal information and questions into the system.

[0267] A "server" is hardware or software that processes data collected from users, trains machine learning models, and generates and provides optimal answers.

[0268] A "terminal" is a device that provides an interface for a user to input personal information and questions, and communicates with a server.

[0269] "Personal Information" is personal data provided by a user, such as name, age, address, income, etc.

[0270] "Account services" are online services used by users, such as communication services, messaging services, and search services.

[0271] "Preprocessing" refers to the process of filtering out unnecessary information from collected data and normalizing the data.

[0272] A "machine learning model" is an algorithm that learns user needs and behavioral patterns based on collected and pre-processed data.

[0273] A "question" is an input regarding a question or request that a user poses to the system.

[0274] An "optimal answer" is a response to a user's question that is generated based on a machine learning model and takes into account the user's individual needs and emotional state.

[0275] An "emotion engine" is software that analyzes and recognizes the user's emotional state from their input.

[0276] "Filtering" is the process of removing unnecessary information from collected data.

[0277] "Normalization" is the process of standardizing and making consistent data formats.

[0278] "Tone" is the phrasing or mood of a response that reflects the user's emotional state.

[0279] The present invention is a system that collects and integrates personal information and information from multiple account services provided by users, inputs the preprocessed data into a machine learning model for learning, and further recognizes the user's emotions and appropriately uses the results to provide optimal answers to questions from users.

[0280] Data Collection Phase

[0281] The server first collects personal information provided by the user. This personal information includes name, age, address, and income. The device prompts the user to enter this information and sends it to the server. The server then aggregates and collects information from multiple account services (e.g., communication services, messaging services, search services, etc.). To obtain this information, it uses the APIs of each service to obtain the necessary user data. The device provides an interface for the user to enter API keys and authentication information.

[0282] Data preprocessing phase

[0283] The server pre-processes the collected personal and account information, which includes filtering unnecessary information and normalizing the data. This procedure improves the quality of the data and optimizes the performance of the learning model.

[0284] Learning Phase

[0285] The server uses the preprocessed data to train a machine learning model. This training process creates a model that reflects the individual needs and behavioral patterns of the user. The trained model is then stored on the server. For example, TensorFlow, a Python library, can be used for machine learning.

[0286] Emotion Recognition Phase

[0287] The server is equipped with an emotion engine that recognizes the user's emotions. When the user inputs a question, the device sends the input voice or text to the server, which analyzes it and recognizes the user's emotions. The emotion engine analyzes the nuances of the text and the tone of the voice to determine whether the user is excited, calm, or stressed.

[0288] Inquiry Processing Phase

[0289] When a user inputs a one-word question from their device, the device sends the question to the server, which uses the trained model to generate an optimal answer to the question. The generated answer is adjusted based on the user's individual profile as well as the user's emotions recognized by the emotion engine. This adjustment provides an answer that is most appropriate for the user's current mental state.

[0290] Finally, the terminal displays the answer received from the server to the user, and a special user interface allows the user to intuitively understand the answer and ask for more detailed information if necessary.

[0291] Specific examples

[0292] For example, consider a user entering a one-word question like, "I want a new refrigerator." If the emotion engine recognizes that the user is a little excited, the server will generate a response in a tone appropriate to that emotion. For example, the response might be something like, "Thank you for your interest. The perfect refrigerator for you is the YY model from XX. Also, please note that there is a discount available at your local recycling center." This provides clear and helpful information while moderating the user's excitement.

[0293] In this system, the user is given a prompt such as "I want a new refrigerator," and is instructed to "generate an appropriate response if the user is excited." In this way, the system provides an optimal answer that takes the user's emotions into account.

[0294] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0295] Step 1:

[0296] The terminal provides an interface that allows the user to input personal information (such as name, age, address, and income). The data to be input is personal information such as name, age, address, and income. The user inputs this information and sends the data from the terminal to a server. The server receives the input personal information and stores it in a database. This stores the user's basic information in the database.

[0297] Step 2:

[0298] The device provides an interface for the user to enter API keys and authentication information for each account service (communication service, messaging service, search service, etc.). The data entered is the API key and authentication information for each service. The user enters this information and sends it from the device to the server. The server uses the received API key and authentication information to call the API of each service and obtain the necessary user data. The obtained data is stored on the server, which allows information from multiple account services to be integrated.

[0299] Step 3:

[0300] The server preprocesses the collected personal and account information. The input for data preprocessing is the collected raw data, which includes unnecessary information. The server first filters out the unnecessary information to improve the quality of the input data. Next, it normalizes the remaining data and converts it into a consistent format. For example, standardizing the format of addresses ensures data integrity. The preprocessed data is then saved back into the database.

[0301] Step 4:

[0302] The server uses the preprocessed data to train a machine learning model. The input for this step is the high-quality data after preprocessing. The server trains the machine learning model using, for example, the Python library TensorFlow. The learning process incorporates user needs and behavioral patterns. The model after this training is completed is stored on the server. This results in a customized model that corresponds to specific user needs.

[0303] Step 5:

[0304] The terminal provides an interface for inputting questions from the user. The input data is the user's question in text or voice. The user inputs the question and sends it from the terminal to the server. The server passes the question to an emotion engine, which analyzes the nuances of the text or voice to determine the user's emotional state. For example, it analyzes information such as whether the user is excited, calm, or stressed.

[0305] Step 6:

[0306] The server uses the emotional information obtained from the emotion engine and the trained model to generate the optimal answer to the user's question. The input for this step is the user's question and emotional information. The server generates an answer in a tone that matches the user's current emotional state. The generated answer is adjusted based on the user's profile and emotion recognition. This prepares the optimal answer according to the emotion.

[0307] Step 7:

[0308] The server sends the generated optimal answer to the terminal. The terminal receives the answer sent from the server and displays it to the user. The output of this step is the optimal answer to the user's question. A special user interface allows the user to intuitively understand the answer and even request more detailed information. This allows the information the user is looking for to be provided quickly and appropriately.

[0309] (Application example 2)

[0310] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0311] In modern content delivery services, it is important to efficiently recommend content that meets users' needs. However, existing systems simply recommend content without considering the user's individual emotional state, which fails to sufficiently increase user satisfaction. Furthermore, they lack a mechanism for integrating data from multiple account services, analyzing user emotions, and proposing optimal content. Therefore, there is a need for a system that efficiently collects and analyzes users' personal information and information from multiple account services, and recommends content that best suits the user's emotions.

[0312] The identification process by the identification processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for collecting personal information provided by the user, means for integrating and collecting information from multiple account services, means for preprocessing the collected data and inputting it into a learning model, means for training a machine learning model using the preprocessed data, means for analyzing the user's emotions, and means for recommending content based on the results of the emotion analysis. This makes it possible to provide optimal content according to the user's emotional state.

[0313] "User" means an individual or a person associated with an individual who uses the system to provide information.

[0314] "Personal information" refers to data related to the user himself / herself, such as the user's name, age, address, hobbies, preferences, and income.

[0315] "Account service" refers to multiple online services to which a user is registered (e.g., social networking services, video streaming services, music streaming services, etc.).

[0316] A "machine learning model" refers to an algorithm or system that uses collected data to learn and make predictions.

[0317] "Emotion analysis means" refers to a system or algorithm that analyzes text or voice data entered by a user to determine their emotional state (e.g., positive, negative, neutral, etc.).

[0318] "Content" refers to digital content such as videos, music, and books, and indicates data and services provided to users.

[0319] "Preprocessing" refers to the process of filtering and normalizing collected data to make it suitable for machine learning models.

[0320] "API" refers to the application programming interface provided by each service, which allows other systems and applications to use the service's functions and data.

[0321] "Server" refers to a computer or network system that collects, processes, analyzes information from users, and returns the results.

[0322] "Recommendation" refers to the act of suggesting the most suitable content based on the user's past behavioral data and current emotional state.

[0323] The present invention is a system that collects and integrates personal information and information from multiple account services provided by users, inputs the preprocessed data into a machine learning model for learning, and then recognizes the user's emotions and appropriately uses the results to provide optimal answers to questions and requests from users.

[0324] System configuration and operation

[0325] Data Collection Phase

[0326] The server first collects personal information provided by the user, including name, age, address, hobbies, preferences, income, etc. The terminal prompts the user to enter this information and sends it to the server.

[0327] Next, the server aggregates and collects information from multiple account services (e.g., social networking sites, video streaming services, music streaming services, etc.). To obtain this information, it uses the APIs of each service to obtain the necessary user data. The device provides an interface for the user to enter their API key and authentication information.

[0328] Data preprocessing phase

[0329] The server pre-processes the collected personal and account information, which includes filtering unnecessary information and normalizing the data. This procedure improves the quality of the data and optimizes the performance of the learning model.

[0330] Learning Phase

[0331] The server uses the preprocessed data to train a machine learning model. This training process creates a model that reflects the individual needs and behavioral patterns of the user. Once the model is trained, it is stored on the server.

[0332] Emotion Recognition Phase

[0333] The server is equipped with an emotion engine that recognizes the user's emotions. When a user inputs a question or request, the device sends the input voice or text to the server, which analyzes it and recognizes the user's emotions. The emotion engine analyzes the nuances of the text and the tone of the voice to determine whether the user is excited, calm, or stressed.

[0334] Inquiry Processing Phase

[0335] When a user inputs a question or request from their device, the device sends the question or request to the server. The server uses the trained model to generate the optimal answer to the question. The generated answer is adjusted based on the individual user profile and the user's emotions recognized by the emotion engine. This adjustment provides an answer that is best suited to the user's current mental state.

[0336] Examples and prompts

[0337] For example, if a user inputs "I'm tired, please recommend some relaxing music," the emotion engine will recognize that the user is seeking relaxation. The server will then select content appropriate for that emotional state, such as healing music or a relaxing movie, and recommend it to the user in the form of "Thank you for your hard work. How about this relaxing music?"

[0338] Example prompt sentence:

[0339] "Username: Yamada Taro, Age: 30, Address: Shinjuku-ku, Tokyo. He asked for recommendations for new movies and music. He seems to be stressed at the moment."

[0340] This system will recommend optimal content based on the user's emotional state, providing a more personalized experience and significantly improving user satisfaction.

[0341] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0342] Step 1: Collect user information

[0343] Users use their devices to input and collect personal information such as their name, age, address, hobbies, and preferences. This information is sent to the server.

[0344] Input: Personal information such as name, age, address, hobbies, preferences, etc.

[0345] Output: Personal information data stored on the server

[0346] Specific operation: The terminal sends the input data, and the server stores it in the database.

[0347] Step 2: Integrated collection of account information

[0348] The user enters an API key and authentication information into their device to obtain information such as playlists and viewing history, and sends that information to the server. The server then obtains the necessary data from multiple account services (e.g., social networking sites, video streaming services) via the API.

[0349] Input: API key or authentication information

[0350] Output: User data obtained from the account service

[0351] Specific operation: The device sends the user's authentication information, and the server calls various APIs to collect data.

[0352] Step 3: Preprocessing the data

[0353] The server pre-processes the collected personal and account information, which includes filtering unnecessary information and normalizing the data.

[0354] Input: Collected user personal and account information

[0355] Output: Preprocessed data

[0356] Specific operation: The server performs data cleansing and normalization of the data.

[0357] Step 4: Training the machine learning model

[0358] The server uses the preprocessed data to train a machine learning model, which is then adjusted based on the user's behavioral patterns and preferences and stored on the server.

[0359] Input: Preprocessed data

[0360] Output: A trained machine learning model

[0361] What it does: The server splits the data into training sets and applies machine learning algorithms to build models.

[0362] Step 5: Emotion Recognition

[0363] The user inputs a question or request into the device, which then sends the input voice or text to the server, where the emotion engine analyzes it to recognize the user's emotions.

[0364] Input: Voice or text of the user's question or request

[0365] Output: Perceived emotional state (e.g., positive, negative, neutral)

[0366] Specific operation: The server uses an emotion analysis algorithm to analyze the user's text and voice data and determine their emotional state.

[0367] Step 6: Recommending the best content

[0368] The server uses the results of the trained machine learning model and emotion engine to recommend the most suitable content to the user, and the recommended content is tailored based on the user's current emotional state.

[0369] Input: Perceived emotional state and user profile

[0370] Output: Recommended content

[0371] Specific operation: The server uses a machine learning model to select the optimal content based on user data and emotional data, and sends the results to the device.

[0372] Step 7: Provide to users

[0373] The terminal displays the most suitable content provided by the server to the user, who can then view the recommended content and request more detailed information.

[0374] Input: Recommended content provided by the server

[0375] Output: Content information displayed on the device

[0376] Specific operation: The device uses a user interface to display the recommended content and allow the user to access it.

[0377] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0378] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0379] In the above embodiment, an example in which the specific process is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific process may be performed by the smart device 14.

[0380] [Second embodiment]

[0381] FIG. 3 shows an example of the configuration of a data processing system 210 according to the second embodiment.

[0382] 3, the data processing system 210 includes the data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0383] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0384] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, and the camera 42 are also connected to the bus 52.

[0385] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0386] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0387] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0388] Fig. 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Fig. 4, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0389] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0390] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0391] In the smart glasses 214, the processor 46 performs the reception output process. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0392] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal."

[0393] The present invention is a system that collects and integrates personal information and information from multiple account services provided by users, inputs the preprocessed data into a machine learning model, and trains it to provide optimal answers to questions from users.

[0394] System configuration and operation

[0395] Data Collection Phase

[0396] The server first collects personal information provided by the user, including name, age, address, income, etc. The terminal prompts the user to enter this information and sends it to the server.

[0397] The server then aggregates and collects information from multiple account services (e.g., communication services, messaging services, search services, etc.). To obtain this information, it uses the APIs of each service to obtain the necessary user data. The device provides an interface for the user to enter their API key and authentication information.

[0398] Data preprocessing phase

[0399] The server preprocesses the collected data and converts it into a format suitable for the learning model. Preprocessing includes filtering unnecessary information and normalizing the data. This procedure improves the quality of the data and optimizes the performance of the learning model.

[0400] Learning Phase

[0401] The server uses the preprocessed data to train a machine learning model, which then creates a model that reflects the individual needs and behavioral patterns of the user. The server then stores the trained model and uses it for subsequent query processing.

[0402] Inquiry Processing Phase

[0403] When a user types a one-word question into their device, the device sends the question to the server, which uses the trained model to generate the best answer for the question. The generated answer is customized based on the individual user profile, so it can meet the user's needs with high accuracy.

[0404] Finally, the terminal displays the answer received from the server to the user, and a special user interface allows the user to intuitively understand the answer and ask for more detailed information if necessary.

[0405] Specific examples

[0406] For example, consider a case where a user is looking for a new refrigerator. The user enters a one-word question such as, "I want a new refrigerator." When the device sends this question to the server, the server will suggest the optimal refrigerator model based on the user's financial situation and living environment.

[0407] The server selects the refrigerator with the best price and energy efficiency, taking into account the user's income and the electricity rates in the area where they live, for example. It also provides information on local recycling services and discount campaigns. In this way, users can obtain information on the refrigerator that is best suited to them by simply asking a question.

[0408] The above is an embodiment of the present invention, and the specific configuration and operation of the system have been described. This system allows users to easily obtain optimal information, thereby providing high convenience.

[0409] The processing flow will be explained below.

[0410] Step 1:

[0411] The user enters personal information, including name, age, address, income, etc. The entered information is collected by the terminal and sent to the server.

[0412] Step 2:

[0413] The server aggregates and collects information from multiple account services (e.g., communication services, messaging services, search services). The device provides an interface for users to enter API keys and authentication information. The server obtains user data using the APIs of each service.

[0414] Step 3:

[0415] The server preprocesses the collected personal and account information, which includes filtering out unnecessary information and normalizing the data, converting it into a format suitable for the learning model.

[0416] Step 4:

[0417] The server uses the preprocessed data to train a machine learning model. This training process creates a model that reflects the individual needs and behavioral patterns of the user. Once trained, the model is stored on the server.

[0418] Step 5:

[0419] The user inputs a one-word question into the terminal, for example, "I want a new refrigerator." The terminal then sends this question to the server.

[0420] Step 6:

[0421] The server uses the trained model to generate optimal answers to user questions, and suggests optimal products and services based on the user's profile and the content of the question.

[0422] Step 7:

[0423] The terminal displays the answer received from the server to the user, which may include, for example, recommended refrigerator models and prices, local recycling services, and discount campaign information.

[0424] The above is the specific processing flow of the system program. This allows users to easily obtain the most appropriate information, providing a high level of convenience.

[0425] Example 1

[0426] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0427] In conventional systems, when integrating and using information provided by users with information from multiple account services, it has been difficult to ensure data accuracy and consistency, and to provide optimal information to individual users. Furthermore, there has been a need to efficiently train machine learning models and generate fast, highly accurate answers to user questions. The present invention aims to solve these problems and provide users with highly accurate, optimal information.

[0428] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0429] In this invention, the server includes: means for collecting personal information provided by a user; means for integrating and collecting information from multiple account services; means for preprocessing the collected data and inputting it into a learning model; means for training a machine learning model using the preprocessed data; means for generating optimal answers to questions from users; means for a terminal to acquire authentication information from the user and send it to the server; means for the server to acquire user data using APIs of each service; means for the server to normalize data and filter unnecessary data; means for training the preprocessed data using a machine learning algorithm and saving the model; and means for accepting user questions, generating answers using the saved model, and sending them to the terminal. This makes it possible to provide users with quick and accurate answers that meet their individual needs.

[0430] A "user" is an individual or group that uses the system and provides personal information, authentication information, etc.

[0431] A "server" is a computer system that collects, stores, and processes information provided by users and trains and executes machine learning models.

[0432] A "terminal" is a device that a user uses to enter information or submit a question, and is an interface that communicates with a server.

[0433] "Personal information" refers to basic information about a user, including data such as name, age, address, and income.

[0434] "Account services" refer to various online services used by users, including communication services, messaging services, search services, and the like.

[0435] "API" stands for Application Program Interface, a set of rules and protocols for exchanging data between different software programs.

[0436] "Data normalization" is the process of converting collected data into a consistent format, a process undertaken to improve data quality.

[0437] "Data filtering" is the process of removing information from a dataset that is not useful to a machine learning model.

[0438] A "machine learning model" is an algorithm or system that uses collected and preprocessed data to learn and perform a specific task.

[0439] A "learning algorithm" is a mathematical or statistical method for using data to train a machine learning model and improve its performance.

[0440] "Authentication information" refers to information required for a user to access each account service, including API keys and passwords.

[0441] A "question" is an inquiry that a user inputs into the system, and is information that triggers the server to generate the most appropriate answer.

[0442] An "answer" is a response generated by the server to a user's question, customized based on the user's profile and data.

[0443] MODE FOR CARRYING OUT THE INVENTION

[0444] The present invention is a system that collects and integrates personal information and information from multiple account services provided by users, inputs the preprocessed data into a machine learning model, and trains it to provide optimal answers to questions from users.

[0445] System Configuration

[0446] The system mainly consists of a server, a terminal, and a user. Each component is explained in detail below.

[0447] Data Collection Phase

[0448] The user uses the device to enter personal information, such as name, age, address, and income. The entered information is sent to the server via the device. The server then retrieves the user data using APIs of multiple account services (e.g., communication services, messaging services, search services, etc.). The device provides an interface for the user to enter the necessary API keys and authentication information. Protocols such as REST API and OAuth are used to collect this type of data.

[0449] Data preprocessing phase

[0450] The server preprocesses the collected data, converting it into a format suitable for the learning model. Preprocessing includes filtering out unnecessary information and normalizing the data. For example, it converts address information into a standard format and standardizes the currency unit of income information. Techniques used include regular expressions and statistical methods.

[0451] Learning Phase

[0452] The server uses the preprocessed data to train a machine learning model. For training, a machine learning framework such as TensorFlow or PyTorch is used. This creates a model that reflects the individual needs and behavioral patterns of the user. The trained model is saved in the server's file system or database. Common save formats include model.h5 and model.pt.

[0453] Inquiry Processing Phase

[0454] When a user inputs a question from their device, the device sends the question to the server, which uses the trained model to generate the optimal answer to the question. For example, if a user asks, "I want a new refrigerator," the server will suggest the optimal refrigerator model based on the user's financial situation and living environment. The answer is sent to the device and displayed to the user.

[0455] Specific examples

[0456] For example, consider a case where a user inputs "I want a new refrigerator." When the device sends this question to the server, the server will suggest the optimal refrigerator model based on the user's personal information and collected account service data. Taking into account income and local electricity rates, the server will select the refrigerator with the best price range and energy efficiency, and also provide information on local recycling services and discount campaigns.

[0457] Here is an example prompt:

[0458] User: I want a new refrigerator.

[0459] Terminal: Sends user questions to the server.

[0460] Server: Recommends the optimal refrigerator model based on the user's financial situation and living environment. For example, it selects the refrigerator with the best price range and energy efficiency, taking into account income and local electricity rates, and also provides information on local recycling services and discount campaigns.

[0461] Terminal: Shows the suggested refrigerator information to the user.

[0462] This system allows users to easily obtain the most appropriate information, providing a high level of convenience.

[0463] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0464] System processing steps

[0465] Step 1: Collect user information

[0466] Input: The user enters personal information into the device.

[0467] Specific behavior: The user adds personal information such as name, age, address, income, etc. to the input form. Once the input is complete, the user clicks the "Submit" button.

[0468] Data processing: The terminal sends the entered personal information to the server via the REST API.

[0469] Output: The server receives the user's personal information and stores it in a database.

[0470] Step 2: Integrate your account service information

[0471] Input: The user enters the authentication information for each account service into the device.

[0472] Specific operation: The user enters the API key and authentication information into the input form and clicks the "Submit" button.

[0473] Data processing: The terminal sends authentication information to the server via REST API, and the server retrieves user data using the API of each service.

[0474] Output: The server retrieves user data from each account service and integrates it into a database.

[0475] Step 3: Normalize the data

[0476] Input: Personal information and account service information collected by the server

[0477] Specific operation: The server converts the address information into a unified format and unifies the currency unit of the income information.

[0478] Data manipulation: Using regular expressions and statistical techniques to transform data into a consistent format.

[0479] Output: The server generates the normalized data and stores it in a database.

[0480] Step 4: Filtering out unnecessary data

[0481] Input: Server-normalized data

[0482] Specific operation: The server filters out unnecessary information (e.g., phone numbers, email addresses).

[0483] Data processing: The server uses scripts to automatically remove unnecessary fields.

[0484] Output: The server generates the filtered data and stores it in a database.

[0485] Step 5: Prepare the training data

[0486] Input: Server filtered data

[0487] What it does: The server converts the data into a format suitable for the machine learning model.

[0488] Data processing: The preprocessed data is formatted and converted into a format that is easy for machine learning algorithms to use (e.g., CSV or JSON).

[0489] Output: The server generates a dataset that can be trained on.

[0490] Step 6: Training the model

[0491] Input: Training data prepared by the server

[0492] Specific operation: The server trains machine learning models using TensorFlow or PyTorch.

[0493] Data processing: The server inputs the training data into the algorithm to train the model.

[0494] Output: The server generates the trained model and saves it to the file system (in the format model.h5 or model.pt).

[0495] Step 7: Ask questions

[0496] Input: The user types a question into the terminal

[0497] Specific behavior: The user enters a question into the input form and clicks the "Submit" button.

[0498] Data processing: The terminal sends the question to the server via the REST API.

[0499] Output: The server receives the query and records it in a database for processing.

[0500] Step 8: Generate an answer

[0501] Input: The question received by the server and the trained model

[0502] What it does: The server uses the trained model to analyze the question and generate the best answer.

[0503] Data Calculation: The server analyzes the question and generates an answer based on the user profile and model.

[0504] Output: The server records the generated answer in a database and sends it to the terminal.

[0505] Step 9: View your answers

[0506] Input: The answer sent by the server

[0507] Specific operation: The terminal displays the answer received from the server to the user.

[0508] Data processing: The terminal embeds the response data into a template and converts it into a format for display.

[0509] Output: The user can check the answer on the device.

[0510] The above are the specific processing steps of this system, which allows users to easily obtain optimal information quickly and accurately.

[0511] (Application example 1)

[0512] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0513] Today, many users use multiple electronic payment services, and checking the benefits and campaign information for each service individually is a significant burden. It is also difficult to provide accurate savings advice based on the user's consumption patterns. As a result, many users miss out on appropriate savings methods and useful campaign information. Therefore, there is a need for a system that can integrate and collect information from users' personal information and multiple account services, and provide optimal savings advice and campaign information based on each user's individual consumption patterns.

[0514] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0515] In this invention, the server includes means for collecting personal information provided by a user, means for integrating and collecting information from multiple account services, means for preprocessing the collected data and inputting it into a learning model, means for training a machine learning model using the preprocessed data, means for analyzing the user's consumption pattern and generating optimal saving methods and campaign information, and means for providing the generated saving methods and campaign information to the user. This enables the user to integrate and analyze data from multiple information sources and easily obtain accurate saving advice and useful campaign information based on their own consumption pattern.

[0516] "User" refers to an individual or corporation that uses the system.

[0517] "Personal Information" refers to information that can be used to identify a specific individual, such as a user's name, age, address, or income.

[0518] "Account Service" refers to a service that provides information related to accounts created by users with electronic payment services and other online services.

[0519] "Means for integrating and collecting information" refers to the process of centralizing and collecting user data from multiple account services.

[0520] "Preprocessing" refers to the process of filtering unnecessary information from collected data and normalizing the data.

[0521] "Means for inputting to a learning model" refers to a method for feeding preprocessed data to a machine learning algorithm for learning.

[0522] A "machine learning model" refers to an algorithm that uses large amounts of data to learn specific patterns and relationships and then makes predictions and classifications for new data.

[0523] "Consumption patterns" refers to information obtained by identifying specific spending tendencies or behavioral patterns from a user's past spending history.

[0524] "Savings tips" refer to specific measures or techniques suggested to help users reduce their spending.

[0525] "Campaign Information" refers to information regarding discounts and benefits offered by electronic payment services and related services.

[0526] "Means of generation" refers to the process of using a machine learning model to create optimal savings methods and campaign information based on a user's individual information.

[0527] The "means of providing" refers to a method of communicating the generated savings methods and campaign information to users.

[0528] The present invention is a system that integrates personal information collected from users and information from multiple account services, and uses the preprocessed data to train a machine learning model. The method by which this system provides optimal answers to user questions is described in detail below.

[0529] System configuration and operation

[0530] Data Collection Phase

[0531] The server first collects personal information provided by the user, including name, age, address, income, etc. The terminal prompts the user to enter this information and sends it to the server.

[0532] The server then aggregates and collects information from multiple account services (such as electronic payment services and other online services). To obtain this information, it uses the API of each service to obtain the necessary user data. The device provides an interface for the user to enter their API key and authentication information.

[0533] Data preprocessing phase

[0534] The server preprocesses the collected data and converts it into a format suitable for the learning model. Preprocessing includes filtering unnecessary information and normalizing the data. This procedure improves the quality of the data and optimizes the performance of the learning model.

[0535] Learning Phase

[0536] The server uses the preprocessed data to train a machine learning model, which then creates a model that reflects the individual user's consumption patterns. The server then stores the trained model and uses it for subsequent query processing.

[0537] Inquiry Processing Phase

[0538] When a user inputs a question about saving money or campaign information on their device, the device sends the question to the server, which uses the trained model to generate the optimal answer to the question. The generated answer is customized based on the individual user profile, so it can meet the user's needs with high accuracy.

[0539] Finally, the terminal displays the answer received from the server to the user, and a special user interface allows the user to intuitively understand the answer and ask for more detailed information if necessary.

[0540] Specific examples

[0541] For example, consider a case where a user wants to know about their future spending patterns. The user inputs the question, "Please tell me about your future spending patterns." When the device sends this question to the server, the server will suggest optimal savings methods and useful campaign information based on the user's income and past spending data.

[0542] Specific prompt examples:

[0543] Prompt: Tell us about your future spending patterns

[0544] Response generation context:

[0545] Username: Taro

[0546] Age: 30

[0547] Income: 500,000 yen per month

[0548] Historical spending data: [Provided in list format]

[0549] Generated response:

[0550] Basic money saving advice

[0551] Electronic payment service benefits and campaign information

[0552] In this way, the system of the present invention provides useful information to the user and achieves high convenience.

[0553] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0554] Step 1:

[0555] Data collection

[0556] The server first collects personal information (such as name, age, address, and income) from the user via the device. This input data is sent to the server in JSON format.

[0557] Next, the server obtains the user's transaction history and account information through the APIs of multiple electronic payment services, and the user enters the API key and authentication information on the terminal.

[0558] The server integrates the acquired data and stores it as a single dataset. The input is user data from each service, and the output is the integrated dataset.

[0559] Step 2:

[0560] Data Preprocessing

[0561] The server then filters unnecessary information from the combined data set and extracts only the information needed, such as past transaction history and income data.

[0562] The extracted data is then normalized, which improves the quality of the data and allows the learning model to analyze it more efficiently.

[0563] The preprocessed data is converted into a format suitable for the machine learning model. The input is the unfiltered dataset, and the output is the preprocessed dataset.

[0564] Step 3:

[0565] Training a machine learning model

[0566] The server uses the preprocessed data to train a machine learning model, using algorithms such as Linear Regression and Random Forest.

[0567] The training process creates a model that reflects your income and spending patterns. The input is the preprocessed dataset, and the output is the trained model.

[0568] The trained model is stored on the server and used for subsequent query processing.

[0569] Step 4:

[0570] Query Processing

[0571] The user inputs a question into the terminal. For example, a simple question such as "Please tell me about your future spending patterns" is input.

[0572] The terminal sends a question from the user to the server.

[0573] Step 5:

[0574] Generating optimal answers

[0575] The server uses the trained model to generate the best answer to the user's question. Based on this prompt, the model generates an answer based on the user's consumption patterns.

[0576] The generated answers are customized for each user and include specific savings methods and campaign information. The input is the user's question and the trained model, and the output is the optimal answer.

[0577] Step 6:

[0578] Providing answers

[0579] The server sends the generated response to the terminal.

[0580] The terminal displays the answer received from the server to the user, who can then consider specific actions based on the presented information. The input is the optimal answer, and the output is the displayed information.

[0581] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[0582] The present invention is a system that collects and integrates personal information and information from multiple account services provided by users, inputs the preprocessed data into a machine learning model for learning, and further recognizes the user's emotions and appropriately uses the results to provide optimal answers to questions from users.

[0583] System configuration and operation

[0584] Data Collection Phase

[0585] The server first collects personal information provided by the user, such as name, age, address, and income, and the terminal prompts the user to enter this information and sends it to the server.

[0586] The server then aggregates and collects information from multiple account services (e.g., communication services, messaging services, search services, etc.). To obtain this information, it uses the APIs of each service to obtain the necessary user data. The device provides an interface for the user to enter their API key and authentication information.

[0587] Data preprocessing phase

[0588] The server pre-processes the collected personal and account information, which includes filtering unnecessary information and normalizing the data. This procedure improves the quality of the data and optimizes the performance of the learning model.

[0589] Learning Phase

[0590] The server uses the preprocessed data to train a machine learning model. This training process creates a model that reflects the individual needs and behavioral patterns of the user. Once the model is trained, it is stored on the server.

[0591] Emotion Recognition Phase

[0592] The server is equipped with an emotion engine that recognizes the user's emotions. When the user inputs a question, the device sends the input voice or text to the server, which analyzes it and recognizes the user's emotions. The emotion engine analyzes the nuances of the text and the tone of the voice to determine whether the user is excited, calm, or stressed.

[0593] Inquiry Processing Phase

[0594] When a user inputs a one-word question from their device, the device sends the question to the server, which uses the trained model to generate an optimal answer to the question. The generated answer is adjusted based on the user's individual profile as well as the user's emotions recognized by the emotion engine. This adjustment provides an answer that is most appropriate for the user's current mental state.

[0595] Finally, the terminal displays the answer received from the server to the user, and a special user interface allows the user to intuitively understand the answer and ask for more detailed information if necessary.

[0596] Specific examples

[0597] For example, imagine a user is looking for a new refrigerator. The user enters a one-word question: "I want a new refrigerator." If the emotion engine detects that the user is a little excited, the server generates a response in a tone appropriate to that emotion. For example, the response might be something like, "Thank you for your interest. The perfect refrigerator for you is the YY model from XX. Also, please note that there is a discount available at your local recycling center." This provides clear and helpful information while moderating the user's excitement.

[0598] In this way, by combining emotion recognition functionality, the present invention makes it possible to provide information tailored to individual user needs with even greater precision. This allows users to obtain the information most suited to them without stress, simply by asking a simple question, and enjoys high convenience.

[0599] The processing flow will be explained below.

[0600] Step 1:

[0601] The user enters personal information, including name, age, address, income, etc. The entered information is collected by the terminal and sent to the server.

[0602] Step 2:

[0603] The server collects and integrates information from multiple account services (e.g., communication services, messaging services, search services). The device provides an interface for the user to enter API keys and authentication information, and the server obtains user data using the APIs of each service.

[0604] Step 3:

[0605] The server preprocesses the collected personal and account information. This preprocessing includes filtering out unnecessary information and normalizing the data. This preprocessing improves the quality of the data and converts it into a format suitable for the learning model.

[0606] Step 4:

[0607] The server uses the preprocessed data to train a machine learning model. This training process creates a model that reflects the individual needs and behavioral patterns of the user. Once trained, the model is stored on the server.

[0608] Step 5:

[0609] The user inputs a question into the terminal. For example, the user inputs a question such as "I want a new refrigerator." The terminal then transmits the input text or voice data to the server.

[0610] Step 6:

[0611] The server uses an emotion engine to analyze the user's input text and voice to recognize the user's emotions. The emotion engine analyzes, for example, the nuances of the text and the tone of the voice to determine whether the user is excited, calm, or stressed.

[0612] Step 7:

[0613] The server uses the trained model to generate the best answer to the user's question, and the emotion engine adjusts the answer based on the user's emotions. For example, if the user is excited, the server creates an answer with a calming tone.

[0614] Step 8:

[0615] The server sends the generated answer to the terminal, which displays the answer to the user. Specifically, the answer includes information about the suggested products and special offers.

[0616] Step 9:

[0617] The user checks the answers displayed on the device and requests more detailed information if necessary. This process allows the user to obtain the information they need quickly and accurately without stress.

[0618] The above is the specific processing flow of the system. With this system, users can ask a simple question and get a highly accurate answer that is best suited to their individual needs.

[0619] Example 2

[0620] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0621] Conventional user support systems struggle to generate appropriate answers based on the user's individual needs. Furthermore, they are unable to generate answers that take the user's emotional state into account, which can result in a poor user experience. Furthermore, insufficient preprocessing of collected information results in suboptimal performance of machine learning models. Therefore, there is a need for a system that can provide emotionally sensitive answers while addressing the user's individual needs.

[0622] The identification process by the identification processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means. In this invention, the server includes a means for recognizing a user's emotion, a means for adjusting a response based on the user's emotion, a means for collecting personal information provided by the user, a means for integrating and collecting information from multiple account services, a means for preprocessing the collected data and inputting it into a machine learning model, a means for training the machine learning model using the preprocessed data, a means for generating an optimal answer to a question from the user, and a means for providing the generated answer to the user. This makes it possible to provide an optimal answer that takes into account the individual needs and emotional state of the user.

[0623] A "user" is an entity that inputs personal information and questions into the system.

[0624] A "server" is hardware or software that processes data collected from users, trains machine learning models, and generates and provides optimal answers.

[0625] A "terminal" is a device that provides an interface for a user to input personal information and questions, and communicates with a server.

[0626] "Personal Information" is personal data provided by a user, such as name, age, address, income, etc.

[0627] "Account services" are online services used by users, such as communication services, messaging services, and search services.

[0628] "Preprocessing" refers to the process of filtering out unnecessary information from collected data and normalizing the data.

[0629] A "machine learning model" is an algorithm that learns user needs and behavioral patterns based on collected and pre-processed data.

[0630] A "question" is an input regarding a question or request that a user poses to the system.

[0631] An "optimal answer" is a response to a user's question that is generated based on a machine learning model and takes into account the user's individual needs and emotional state.

[0632] An "emotion engine" is software that analyzes and recognizes the user's emotional state from their input.

[0633] "Filtering" is the process of removing unnecessary information from collected data.

[0634] "Normalization" is the process of standardizing and making consistent data formats.

[0635] "Tone" is the phrasing or mood of a response that reflects the user's emotional state.

[0636] The present invention is a system that collects and integrates personal information and information from multiple account services provided by users, inputs the preprocessed data into a machine learning model for learning, and further recognizes the user's emotions and appropriately uses the results to provide optimal answers to questions from users.

[0637] Data Collection Phase

[0638] The server first collects personal information provided by the user. This personal information includes name, age, address, and income. The device prompts the user to enter this information and sends it to the server. The server then aggregates and collects information from multiple account services (e.g., communication services, messaging services, search services, etc.). To obtain this information, it uses the APIs of each service to obtain the necessary user data. The device provides an interface for the user to enter API keys and authentication information.

[0639] Data preprocessing phase

[0640] The server pre-processes the collected personal and account information, which includes filtering unnecessary information and normalizing the data. This procedure improves the quality of the data and optimizes the performance of the learning model.

[0641] Learning Phase

[0642] The server uses the preprocessed data to train a machine learning model. This training process creates a model that reflects the individual needs and behavioral patterns of the user. The trained model is then stored on the server. For example, TensorFlow, a Python library, can be used for machine learning.

[0643] Emotion Recognition Phase

[0644] The server is equipped with an emotion engine that recognizes the user's emotions. When the user inputs a question, the device sends the input voice or text to the server, which analyzes it and recognizes the user's emotions. The emotion engine analyzes the nuances of the text and the tone of the voice to determine whether the user is excited, calm, or stressed.

[0645] Inquiry Processing Phase

[0646] When a user inputs a one-word question from their device, the device sends the question to the server, which uses the trained model to generate an optimal answer to the question. The generated answer is adjusted based on the user's individual profile as well as the user's emotions recognized by the emotion engine. This adjustment provides an answer that is most appropriate for the user's current mental state.

[0647] Finally, the terminal displays the answer received from the server to the user, and a special user interface allows the user to intuitively understand the answer and ask for more detailed information if necessary.

[0648] Specific examples

[0649] For example, consider a user entering a one-word question like, "I want a new refrigerator." If the emotion engine recognizes that the user is a little excited, the server will generate a response in a tone appropriate to that emotion. For example, the response might be something like, "Thank you for your interest. The perfect refrigerator for you is the YY model from XX. Also, please note that there is a discount available at your local recycling center." This provides clear and helpful information while moderating the user's excitement.

[0650] In this system, the user is given a prompt such as "I want a new refrigerator," and is instructed to "generate an appropriate response if the user is excited." In this way, the system provides an optimal answer that takes the user's emotions into account.

[0651] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0652] Step 1:

[0653] The terminal provides an interface that allows the user to input personal information (such as name, age, address, and income). The data to be input is personal information such as name, age, address, and income. The user inputs this information and sends the data from the terminal to a server. The server receives the input personal information and stores it in a database. This stores the user's basic information in the database.

[0654] Step 2:

[0655] The device provides an interface for the user to enter API keys and authentication information for each account service (communication service, messaging service, search service, etc.). The data entered is the API key and authentication information for each service. The user enters this information and sends it from the device to the server. The server uses the received API key and authentication information to call the API of each service and obtain the necessary user data. The obtained data is stored on the server, which allows information from multiple account services to be integrated.

[0656] Step 3:

[0657] The server preprocesses the collected personal and account information. The input for data preprocessing is the collected raw data, which includes unnecessary information. The server first filters out the unnecessary information to improve the quality of the input data. Next, it normalizes the remaining data and converts it into a consistent format. For example, standardizing the format of addresses ensures data integrity. The preprocessed data is then saved back into the database.

[0658] Step 4:

[0659] The server uses the preprocessed data to train a machine learning model. The input for this step is the high-quality data after preprocessing. The server trains the machine learning model using, for example, the Python library TensorFlow. The learning process incorporates user needs and behavioral patterns. The model after this training is completed is stored on the server. This results in a customized model that corresponds to specific user needs.

[0660] Step 5:

[0661] The terminal provides an interface for inputting questions from the user. The input data is the user's question in text or voice. The user inputs the question and sends it from the terminal to the server. The server passes the question to an emotion engine, which analyzes the nuances of the text or voice to determine the user's emotional state. For example, it analyzes information such as whether the user is excited, calm, or stressed.

[0662] Step 6:

[0663] The server uses the emotional information obtained from the emotion engine and the trained model to generate the optimal answer to the user's question. The input for this step is the user's question and emotional information. The server generates an answer in a tone that matches the user's current emotional state. The generated answer is adjusted based on the user's profile and emotion recognition. This prepares the optimal answer according to the emotion.

[0664] Step 7:

[0665] The server sends the generated optimal answer to the terminal. The terminal receives the answer sent from the server and displays it to the user. The output of this step is the optimal answer to the user's question. A special user interface allows the user to intuitively understand the answer and even request more detailed information. This allows the information the user is looking for to be provided quickly and appropriately.

[0666] (Application example 2)

[0667] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0668] In modern content delivery services, it is important to efficiently recommend content that meets users' needs. However, existing systems simply recommend content without considering the user's individual emotional state, which fails to sufficiently increase user satisfaction. Furthermore, they lack a mechanism for integrating data from multiple account services, analyzing user emotions, and proposing optimal content. Therefore, there is a need for a system that efficiently collects and analyzes users' personal information and information from multiple account services, and recommends content that best suits the user's emotions.

[0669] The identification process by the identification processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for collecting personal information provided by the user, means for integrating and collecting information from multiple account services, means for preprocessing the collected data and inputting it into a learning model, means for training a machine learning model using the preprocessed data, means for analyzing the user's emotions, and means for recommending content based on the results of the emotion analysis. This makes it possible to provide optimal content according to the user's emotional state.

[0670] "User" means an individual or a person associated with an individual who uses the system to provide information.

[0671] "Personal information" refers to data related to the user himself / herself, such as the user's name, age, address, hobbies, preferences, and income.

[0672] "Account service" refers to multiple online services to which a user is registered (e.g., social networking services, video streaming services, music streaming services, etc.).

[0673] A "machine learning model" refers to an algorithm or system that uses collected data to learn and make predictions.

[0674] "Emotion analysis means" refers to a system or algorithm that analyzes text or voice data entered by a user to determine their emotional state (e.g., positive, negative, neutral, etc.).

[0675] "Content" refers to digital content such as videos, music, and books, and indicates data and services provided to users.

[0676] "Preprocessing" refers to the process of filtering and normalizing collected data to make it suitable for machine learning models.

[0677] "API" refers to the application programming interface provided by each service, which allows other systems and applications to use the service's functions and data.

[0678] "Server" refers to a computer or network system that collects, processes, analyzes information from users, and returns the results.

[0679] "Recommendation" refers to the act of suggesting the most suitable content based on the user's past behavioral data and current emotional state.

[0680] The present invention is a system that collects and integrates personal information and information from multiple account services provided by users, inputs the preprocessed data into a machine learning model for learning, and then recognizes the user's emotions and appropriately uses the results to provide optimal answers to questions and requests from users.

[0681] System configuration and operation

[0682] Data Collection Phase

[0683] The server first collects personal information provided by the user, including name, age, address, hobbies, preferences, income, etc. The terminal prompts the user to enter this information and sends it to the server.

[0684] Next, the server aggregates and collects information from multiple account services (e.g., social networking sites, video streaming services, music streaming services, etc.). To obtain this information, it uses the APIs of each service to obtain the necessary user data. The device provides an interface for the user to enter their API key and authentication information.

[0685] Data preprocessing phase

[0686] The server pre-processes the collected personal and account information, which includes filtering unnecessary information and normalizing the data. This procedure improves the quality of the data and optimizes the performance of the learning model.

[0687] Learning Phase

[0688] The server uses the preprocessed data to train a machine learning model. This training process creates a model that reflects the individual needs and behavioral patterns of the user. Once the model is trained, it is stored on the server.

[0689] Emotion Recognition Phase

[0690] The server is equipped with an emotion engine that recognizes the user's emotions. When a user inputs a question or request, the device sends the input voice or text to the server, which analyzes it and recognizes the user's emotions. The emotion engine analyzes the nuances of the text and the tone of the voice to determine whether the user is excited, calm, or stressed.

[0691] Inquiry Processing Phase

[0692] When a user inputs a question or request from their device, the device sends the question or request to the server. The server uses the trained model to generate the optimal answer to the question. The generated answer is adjusted based on the individual user profile and the user's emotions recognized by the emotion engine. This adjustment provides an answer that is best suited to the user's current mental state.

[0693] Examples and prompts

[0694] For example, if a user inputs "I'm tired, please recommend some relaxing music," the emotion engine will recognize that the user is seeking relaxation. The server will then select content appropriate for that emotional state, such as healing music or a relaxing movie, and recommend it to the user in the form of "Thank you for your hard work. How about this relaxing music?"

[0695] Example prompt sentence:

[0696] "Username: Yamada Taro, Age: 30, Address: Shinjuku-ku, Tokyo. He asked for recommendations for new movies and music. He seems to be stressed at the moment."

[0697] This system will recommend optimal content based on the user's emotional state, providing a more personalized experience and significantly improving user satisfaction.

[0698] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0699] Step 1: Collect user information

[0700] Users use their devices to input and collect personal information such as their name, age, address, hobbies, and preferences. This information is sent to the server.

[0701] Input: Personal information such as name, age, address, hobbies, preferences, etc.

[0702] Output: Personal information data stored on the server

[0703] Specific operation: The terminal sends the input data, and the server stores it in the database.

[0704] Step 2: Integrated collection of account information

[0705] The user enters an API key and authentication information into their device to obtain information such as playlists and viewing history, and sends that information to the server. The server then obtains the necessary data from multiple account services (e.g., social networking sites, video streaming services) via the API.

[0706] Input: API key or authentication information

[0707] Output: User data obtained from the account service

[0708] Specific operation: The device sends the user's authentication information, and the server calls various APIs to collect data.

[0709] Step 3: Preprocessing the data

[0710] The server pre-processes the collected personal and account information, which includes filtering unnecessary information and normalizing the data.

[0711] Input: Collected user personal and account information

[0712] Output: Preprocessed data

[0713] Specific operation: The server performs data cleansing and normalization of the data.

[0714] Step 4: Training the machine learning model

[0715] The server uses the preprocessed data to train a machine learning model, which is then adjusted based on the user's behavioral patterns and preferences and stored on the server.

[0716] Input: Preprocessed data

[0717] Output: A trained machine learning model

[0718] What it does: The server splits the data into training sets and applies machine learning algorithms to build models.

[0719] Step 5: Emotion Recognition

[0720] The user inputs a question or request into the device, which then sends the input voice or text to the server, where the emotion engine analyzes it to recognize the user's emotions.

[0721] Input: Voice or text of the user's question or request

[0722] Output: Perceived emotional state (e.g., positive, negative, neutral)

[0723] Specific operation: The server uses an emotion analysis algorithm to analyze the user's text and voice data and determine their emotional state.

[0724] Step 6: Recommending the best content

[0725] The server uses the results of the trained machine learning model and emotion engine to recommend the most suitable content to the user, and the recommended content is tailored based on the user's current emotional state.

[0726] Input: Perceived emotional state and user profile

[0727] Output: Recommended content

[0728] Specific operation: The server uses a machine learning model to select the optimal content based on user data and emotional data, and sends the results to the device.

[0729] Step 7: Provide to users

[0730] The terminal displays the most suitable content provided by the server to the user, who can then view the recommended content and request more detailed information.

[0731] Input: Recommended content provided by the server

[0732] Output: Content information displayed on the device

[0733] Specific operation: The device uses a user interface to display the recommended content and allow the user to access it.

[0734] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0735] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0736] In the above embodiment, an example in which the specific processing is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the smart glasses 214.

[0737] [Third embodiment]

[0738] FIG. 5 shows an example of the configuration of a data processing system 310 according to the third embodiment.

[0739] 5, the data processing system 310 includes the data processing device 12 and a headset terminal 314. An example of the data processing device 12 is a server.

[0740] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0741] The headset type terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a display 343. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the display 343 are also connected to the bus 52.

[0742] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0743] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0744] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0745] Fig. 6 shows an example of the main functions of the data processing device 12 and the headset type terminal 314. As shown in Fig. 6, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0746] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0747] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0748] In the headset type terminal 314, a reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0749] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the headset type terminal 314 will be referred to as the "terminal."

[0750] The present invention is a system that collects and integrates personal information and information from multiple account services provided by users, inputs the preprocessed data into a machine learning model, and trains it to provide optimal answers to questions from users.

[0751] System configuration and operation

[0752] Data Collection Phase

[0753] The server first collects personal information provided by the user, including name, age, address, income, etc. The terminal prompts the user to enter this information and sends it to the server.

[0754] The server then aggregates and collects information from multiple account services (e.g., communication services, messaging services, search services, etc.). To obtain this information, it uses the APIs of each service to obtain the necessary user data. The device provides an interface for the user to enter their API key and authentication information.

[0755] Data preprocessing phase

[0756] The server preprocesses the collected data and converts it into a format suitable for the learning model. Preprocessing includes filtering unnecessary information and normalizing the data. This procedure improves the quality of the data and optimizes the performance of the learning model.

[0757] Learning Phase

[0758] The server uses the preprocessed data to train a machine learning model, which then creates a model that reflects the individual needs and behavioral patterns of the user. The server then stores the trained model and uses it for subsequent query processing.

[0759] Inquiry Processing Phase

[0760] When a user types a one-word question into their device, the device sends the question to the server, which uses the trained model to generate the best answer for the question. The generated answer is customized based on the individual user profile, so it can meet the user's needs with high accuracy.

[0761] Finally, the terminal displays the answer received from the server to the user, and a special user interface allows the user to intuitively understand the answer and ask for more detailed information if necessary.

[0762] Specific examples

[0763] For example, consider a case where a user is looking for a new refrigerator. The user enters a one-word question such as, "I want a new refrigerator." When the device sends this question to the server, the server will suggest the optimal refrigerator model based on the user's financial situation and living environment.

[0764] The server selects the refrigerator with the best price and energy efficiency, taking into account the user's income and the electricity rates in the area where they live, for example. It also provides information on local recycling services and discount campaigns. In this way, users can obtain information on the refrigerator that is best suited to them by simply asking a question.

[0765] The above is an embodiment of the present invention, and the specific configuration and operation of the system have been described. This system allows users to easily obtain optimal information, thereby providing high convenience.

[0766] The processing flow will be explained below.

[0767] Step 1:

[0768] The user enters personal information, including name, age, address, income, etc. The entered information is collected by the terminal and sent to the server.

[0769] Step 2:

[0770] The server aggregates and collects information from multiple account services (e.g., communication services, messaging services, search services). The device provides an interface for users to enter API keys and authentication information. The server obtains user data using the APIs of each service.

[0771] Step 3:

[0772] The server preprocesses the collected personal and account information, which includes filtering out unnecessary information and normalizing the data, converting it into a format suitable for the learning model.

[0773] Step 4:

[0774] The server uses the preprocessed data to train a machine learning model. This training process creates a model that reflects the individual needs and behavioral patterns of the user. Once trained, the model is stored on the server.

[0775] Step 5:

[0776] The user inputs a one-word question into the terminal, for example, "I want a new refrigerator." The terminal then sends this question to the server.

[0777] Step 6:

[0778] The server uses the trained model to generate optimal answers to user questions, and suggests optimal products and services based on the user's profile and the content of the question.

[0779] Step 7:

[0780] The terminal displays the answer received from the server to the user, which may include, for example, recommended refrigerator models and prices, local recycling services, and discount campaign information.

[0781] The above is the specific processing flow of the system program. This allows users to easily obtain the most appropriate information, providing a high level of convenience.

[0782] Example 1

[0783] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[0784] In conventional systems, when integrating and using information provided by users with information from multiple account services, it has been difficult to ensure data accuracy and consistency, and to provide optimal information to individual users. Furthermore, there has been a need to efficiently train machine learning models and generate fast, highly accurate answers to user questions. The present invention aims to solve these problems and provide users with highly accurate, optimal information.

[0785] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0786] In this invention, the server includes: means for collecting personal information provided by a user; means for integrating and collecting information from multiple account services; means for preprocessing the collected data and inputting it into a learning model; means for training a machine learning model using the preprocessed data; means for generating optimal answers to questions from users; means for a terminal to acquire authentication information from the user and send it to the server; means for the server to acquire user data using APIs of each service; means for the server to normalize data and filter unnecessary data; means for training the preprocessed data using a machine learning algorithm and saving the model; and means for accepting user questions, generating answers using the saved model, and sending them to the terminal. This makes it possible to provide users with quick and accurate answers that meet their individual needs.

[0787] A "user" is an individual or group that uses the system and provides personal information, authentication information, etc.

[0788] A "server" is a computer system that collects, stores, and processes information provided by users and trains and executes machine learning models.

[0789] A "terminal" is a device that a user uses to enter information or submit a question, and is an interface that communicates with a server.

[0790] "Personal information" refers to basic information about a user, including data such as name, age, address, and income.

[0791] "Account services" refer to various online services used by users, including communication services, messaging services, search services, and the like.

[0792] "API" stands for Application Program Interface, a set of rules and protocols for exchanging data between different software programs.

[0793] "Data normalization" is the process of converting collected data into a consistent format, a process undertaken to improve data quality.

[0794] "Data filtering" is the process of removing information from a dataset that is not useful to a machine learning model.

[0795] A "machine learning model" is an algorithm or system that uses collected and preprocessed data to learn and perform a specific task.

[0796] A "learning algorithm" is a mathematical or statistical method for using data to train a machine learning model and improve its performance.

[0797] "Authentication information" refers to information required for a user to access each account service, including API keys and passwords.

[0798] A "question" is an inquiry that a user inputs into the system, and is information that triggers the server to generate the most appropriate answer.

[0799] An "answer" is a response generated by the server to a user's question, customized based on the user's profile and data.

[0800] MODE FOR CARRYING OUT THE INVENTION

[0801] The present invention is a system that collects and integrates personal information and information from multiple account services provided by users, inputs the preprocessed data into a machine learning model, and trains it to provide optimal answers to questions from users.

[0802] System Configuration

[0803] The system mainly consists of a server, a terminal, and a user. Each component is explained in detail below.

[0804] Data Collection Phase

[0805] The user uses the device to enter personal information, such as name, age, address, and income. The entered information is sent to the server via the device. The server then retrieves the user data using APIs of multiple account services (e.g., communication services, messaging services, search services, etc.). The device provides an interface for the user to enter the necessary API keys and authentication information. Protocols such as REST API and OAuth are used to collect this type of data.

[0806] Data preprocessing phase

[0807] The server preprocesses the collected data, converting it into a format suitable for the learning model. Preprocessing includes filtering out unnecessary information and normalizing the data. For example, it converts address information into a standard format and standardizes the currency unit of income information. Techniques used include regular expressions and statistical methods.

[0808] Learning Phase

[0809] The server uses the preprocessed data to train a machine learning model. For training, a machine learning framework such as TensorFlow or PyTorch is used. This creates a model that reflects the individual needs and behavioral patterns of the user. The trained model is saved in the server's file system or database. Common save formats include model.h5 and model.pt.

[0810] Inquiry Processing Phase

[0811] When a user inputs a question from their device, the device sends the question to the server, which uses the trained model to generate the optimal answer to the question. For example, if a user asks, "I want a new refrigerator," the server will suggest the optimal refrigerator model based on the user's financial situation and living environment. The answer is sent to the device and displayed to the user.

[0812] Specific examples

[0813] For example, consider a case where a user inputs "I want a new refrigerator." When the device sends this question to the server, the server will suggest the optimal refrigerator model based on the user's personal information and collected account service data. Taking into account income and local electricity rates, the server will select the refrigerator with the best price range and energy efficiency, and also provide information on local recycling services and discount campaigns.

[0814] Here is an example prompt:

[0815] User: I want a new refrigerator.

[0816] Terminal: Sends user questions to the server.

[0817] Server: Recommends the optimal refrigerator model based on the user's financial situation and living environment. For example, it selects the refrigerator with the best price range and energy efficiency, taking into account income and local electricity rates, and also provides information on local recycling services and discount campaigns.

[0818] Terminal: Shows the suggested refrigerator information to the user.

[0819] This system allows users to easily obtain the most appropriate information, providing a high level of convenience.

[0820] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0821] System processing steps

[0822] Step 1: Collect user information

[0823] Input: The user enters personal information into the device.

[0824] Specific behavior: The user adds personal information such as name, age, address, income, etc. to the input form. Once the input is complete, the user clicks the "Submit" button.

[0825] Data processing: The terminal sends the entered personal information to the server via the REST API.

[0826] Output: The server receives the user's personal information and stores it in a database.

[0827] Step 2: Integrate your account service information

[0828] Input: The user enters the authentication information for each account service into the device.

[0829] Specific operation: The user enters the API key and authentication information into the input form and clicks the "Submit" button.

[0830] Data processing: The terminal sends authentication information to the server via REST API, and the server retrieves user data using the API of each service.

[0831] Output: The server retrieves user data from each account service and integrates it into a database.

[0832] Step 3: Normalize the data

[0833] Input: Personal information and account service information collected by the server

[0834] Specific operation: The server converts the address information into a unified format and unifies the currency unit of the income information.

[0835] Data manipulation: Using regular expressions and statistical techniques to transform data into a consistent format.

[0836] Output: The server generates the normalized data and stores it in a database.

[0837] Step 4: Filtering out unnecessary data

[0838] Input: Server-normalized data

[0839] Specific operation: The server filters out unnecessary information (e.g., phone numbers, email addresses).

[0840] Data processing: The server uses scripts to automatically remove unnecessary fields.

[0841] Output: The server generates the filtered data and stores it in a database.

[0842] Step 5: Prepare the training data

[0843] Input: Server filtered data

[0844] What it does: The server converts the data into a format suitable for the machine learning model.

[0845] Data processing: The preprocessed data is formatted and converted into a format that is easy for machine learning algorithms to use (e.g., CSV or JSON).

[0846] Output: The server generates a dataset that can be trained on.

[0847] Step 6: Training the model

[0848] Input: Training data prepared by the server

[0849] Specific operation: The server trains machine learning models using TensorFlow or PyTorch.

[0850] Data processing: The server inputs the training data into the algorithm to train the model.

[0851] Output: The server generates the trained model and saves it to the file system (in the format model.h5 or model.pt).

[0852] Step 7: Ask questions

[0853] Input: The user types a question into the terminal

[0854] Specific behavior: The user enters a question into the input form and clicks the "Submit" button.

[0855] Data processing: The terminal sends the question to the server via the REST API.

[0856] Output: The server receives the query and records it in a database for processing.

[0857] Step 8: Generate an answer

[0858] Input: The question received by the server and the trained model

[0859] What it does: The server uses the trained model to analyze the question and generate the best answer.

[0860] Data Calculation: The server analyzes the question and generates an answer based on the user profile and model.

[0861] Output: The server records the generated answer in a database and sends it to the terminal.

[0862] Step 9: View your answers

[0863] Input: The answer sent by the server

[0864] Specific operation: The terminal displays the answer received from the server to the user.

[0865] Data processing: The terminal embeds the response data into a template and converts it into a format for display.

[0866] Output: The user can check the answer on the device.

[0867] The above are the specific processing steps of this system, which allows users to easily obtain optimal information quickly and accurately.

[0868] (Application example 1)

[0869] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[0870] Today, many users use multiple electronic payment services, and checking the benefits and campaign information for each service individually is a significant burden. It is also difficult to provide accurate savings advice based on the user's consumption patterns. As a result, many users miss out on appropriate savings methods and useful campaign information. Therefore, there is a need for a system that can integrate and collect information from users' personal information and multiple account services, and provide optimal savings advice and campaign information based on each user's individual consumption patterns.

[0871] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0872] In this invention, the server includes means for collecting personal information provided by a user, means for integrating and collecting information from multiple account services, means for preprocessing the collected data and inputting it into a learning model, means for training a machine learning model using the preprocessed data, means for analyzing the user's consumption pattern and generating optimal saving methods and campaign information, and means for providing the generated saving methods and campaign information to the user. This enables the user to integrate and analyze data from multiple information sources and easily obtain accurate saving advice and useful campaign information based on their own consumption pattern.

[0873] "User" refers to an individual or corporation that uses the system.

[0874] "Personal Information" refers to information that can be used to identify a specific individual, such as a user's name, age, address, or income.

[0875] "Account Service" refers to a service that provides information related to accounts created by users with electronic payment services and other online services.

[0876] "Means for integrating and collecting information" refers to the process of centralizing and collecting user data from multiple account services.

[0877] "Preprocessing" refers to the process of filtering unnecessary information from collected data and normalizing the data.

[0878] "Means for inputting to a learning model" refers to a method for feeding preprocessed data to a machine learning algorithm for learning.

[0879] A "machine learning model" refers to an algorithm that uses large amounts of data to learn specific patterns and relationships and then makes predictions and classifications for new data.

[0880] "Consumption patterns" refers to information obtained by identifying specific spending tendencies or behavioral patterns from a user's past spending history.

[0881] "Savings tips" refer to specific measures or techniques suggested to help users reduce their spending.

[0882] "Campaign Information" refers to information regarding discounts and benefits offered by electronic payment services and related services.

[0883] "Means of generation" refers to the process of using a machine learning model to create optimal savings methods and campaign information based on a user's individual information.

[0884] The "means of providing" refers to a method of communicating the generated savings methods and campaign information to users.

[0885] The present invention is a system that integrates personal information collected from users and information from multiple account services, and uses the preprocessed data to train a machine learning model. The method by which this system provides optimal answers to user questions is described in detail below.

[0886] System configuration and operation

[0887] Data Collection Phase

[0888] The server first collects personal information provided by the user, including name, age, address, income, etc. The terminal prompts the user to enter this information and sends it to the server.

[0889] The server then aggregates and collects information from multiple account services (such as electronic payment services and other online services). To obtain this information, it uses the API of each service to obtain the necessary user data. The device provides an interface for the user to enter their API key and authentication information.

[0890] Data preprocessing phase

[0891] The server preprocesses the collected data and converts it into a format suitable for the learning model. Preprocessing includes filtering unnecessary information and normalizing the data. This procedure improves the quality of the data and optimizes the performance of the learning model.

[0892] Learning Phase

[0893] The server uses the preprocessed data to train a machine learning model, which then creates a model that reflects the individual user's consumption patterns. The server then stores the trained model and uses it for subsequent query processing.

[0894] Inquiry Processing Phase

[0895] When a user inputs a question about saving money or campaign information on their device, the device sends the question to the server, which uses the trained model to generate the optimal answer to the question. The generated answer is customized based on the individual user profile, so it can meet the user's needs with high accuracy.

[0896] Finally, the terminal displays the answer received from the server to the user, and a special user interface allows the user to intuitively understand the answer and ask for more detailed information if necessary.

[0897] Specific examples

[0898] For example, consider a case where a user wants to know about their future spending patterns. The user inputs the question, "Please tell me about your future spending patterns." When the device sends this question to the server, the server will suggest optimal savings methods and useful campaign information based on the user's income and past spending data.

[0899] Specific prompt examples:

[0900] Prompt: Tell us about your future spending patterns

[0901] Response generation context:

[0902] Username: Taro

[0903] Age: 30

[0904] Income: 500,000 yen per month

[0905] Historical spending data: [Provided in list format]

[0906] Generated response:

[0907] Basic money saving advice

[0908] Electronic payment service benefits and campaign information

[0909] In this way, the system of the present invention provides useful information to the user and achieves high convenience.

[0910] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0911] Step 1:

[0912] Data collection

[0913] The server first collects personal information (such as name, age, address, and income) from the user via the device. This input data is sent to the server in JSON format.

[0914] Next, the server obtains the user's transaction history and account information through the APIs of multiple electronic payment services, and the user enters the API key and authentication information on the terminal.

[0915] The server integrates the acquired data and stores it as a single dataset. The input is user data from each service, and the output is the integrated dataset.

[0916] Step 2:

[0917] Data Preprocessing

[0918] The server then filters unnecessary information from the combined data set and extracts only the information needed, such as past transaction history and income data.

[0919] The extracted data is then normalized, which improves the quality of the data and allows the learning model to analyze it more efficiently.

[0920] The preprocessed data is converted into a format suitable for the machine learning model. The input is the unfiltered dataset, and the output is the preprocessed dataset.

[0921] Step 3:

[0922] Training a machine learning model

[0923] The server uses the preprocessed data to train a machine learning model, using algorithms such as Linear Regression and Random Forest.

[0924] The training process creates a model that reflects your income and spending patterns. The input is the preprocessed dataset, and the output is the trained model.

[0925] The trained model is stored on the server and used for subsequent query processing.

[0926] Step 4:

[0927] Query Processing

[0928] The user inputs a question into the terminal. For example, a simple question such as "Please tell me about your future spending patterns" is input.

[0929] The terminal sends a question from the user to the server.

[0930] Step 5:

[0931] Generating optimal answers

[0932] The server uses the trained model to generate the best answer to the user's question. Based on this prompt, the model generates an answer based on the user's consumption patterns.

[0933] The generated answers are customized for each user and include specific savings methods and campaign information. The input is the user's question and the trained model, and the output is the optimal answer.

[0934] Step 6:

[0935] Providing answers

[0936] The server sends the generated response to the terminal.

[0937] The terminal displays the answer received from the server to the user, who can then consider specific actions based on the presented information. The input is the optimal answer, and the output is the displayed information.

[0938] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[0939] The present invention is a system that collects and integrates personal information and information from multiple account services provided by users, inputs the preprocessed data into a machine learning model for learning, and further recognizes the user's emotions and appropriately uses the results to provide optimal answers to questions from users.

[0940] System configuration and operation

[0941] Data Collection Phase

[0942] The server first collects personal information provided by the user, such as name, age, address, and income, and the terminal prompts the user to enter this information and sends it to the server.

[0943] The server then aggregates and collects information from multiple account services (e.g., communication services, messaging services, search services, etc.). To obtain this information, it uses the APIs of each service to obtain the necessary user data. The device provides an interface for the user to enter their API key and authentication information.

[0944] Data preprocessing phase

[0945] The server pre-processes the collected personal and account information, which includes filtering unnecessary information and normalizing the data. This procedure improves the quality of the data and optimizes the performance of the learning model.

[0946] Learning Phase

[0947] The server uses the preprocessed data to train a machine learning model. This training process creates a model that reflects the individual needs and behavioral patterns of the user. Once the model is trained, it is stored on the server.

[0948] Emotion Recognition Phase

[0949] The server is equipped with an emotion engine that recognizes the user's emotions. When the user inputs a question, the device sends the input voice or text to the server, which analyzes it and recognizes the user's emotions. The emotion engine analyzes the nuances of the text and the tone of the voice to determine whether the user is excited, calm, or stressed.

[0950] Inquiry Processing Phase

[0951] When a user inputs a one-word question from their device, the device sends the question to the server, which uses the trained model to generate an optimal answer to the question. The generated answer is adjusted based on the user's individual profile as well as the user's emotions recognized by the emotion engine. This adjustment provides an answer that is most appropriate for the user's current mental state.

[0952] Finally, the terminal displays the answer received from the server to the user, and a special user interface allows the user to intuitively understand the answer and ask for more detailed information if necessary.

[0953] Specific examples

[0954] For example, imagine a user is looking for a new refrigerator. The user enters a one-word question: "I want a new refrigerator." If the emotion engine detects that the user is a little excited, the server generates a response in a tone appropriate to that emotion. For example, the response might be something like, "Thank you for your interest. The perfect refrigerator for you is the YY model from XX. Also, please note that there is a discount available at your local recycling center." This provides clear and helpful information while moderating the user's excitement.

[0955] In this way, by combining emotion recognition functionality, the present invention makes it possible to provide information tailored to individual user needs with even greater precision. This allows users to obtain the information most suited to them without stress, simply by asking a simple question, and enjoys high convenience.

[0956] The processing flow will be explained below.

[0957] Step 1:

[0958] The user enters personal information, including name, age, address, income, etc. The entered information is collected by the terminal and sent to the server.

[0959] Step 2:

[0960] The server collects and integrates information from multiple account services (e.g., communication services, messaging services, search services). The device provides an interface for the user to enter API keys and authentication information, and the server obtains user data using the APIs of each service.

[0961] Step 3:

[0962] The server preprocesses the collected personal and account information. This preprocessing includes filtering out unnecessary information and normalizing the data. This preprocessing improves the quality of the data and converts it into a format suitable for the learning model.

[0963] Step 4:

[0964] The server uses the preprocessed data to train a machine learning model. This training process creates a model that reflects the individual needs and behavioral patterns of the user. Once trained, the model is stored on the server.

[0965] Step 5:

[0966] The user inputs a question into the terminal. For example, the user inputs a question such as "I want a new refrigerator." The terminal then transmits the input text or voice data to the server.

[0967] Step 6:

[0968] The server uses an emotion engine to analyze the user's input text and voice to recognize the user's emotions. The emotion engine analyzes, for example, the nuances of the text and the tone of the voice to determine whether the user is excited, calm, or stressed.

[0969] Step 7:

[0970] The server uses the trained model to generate the best answer to the user's question, and the emotion engine adjusts the answer based on the user's emotions. For example, if the user is excited, the server creates an answer with a calming tone.

[0971] Step 8:

[0972] The server sends the generated answer to the terminal, which displays the answer to the user. Specifically, the answer includes information about the suggested products and special offers.

[0973] Step 9:

[0974] The user checks the answers displayed on the device and requests more detailed information if necessary. This process allows the user to obtain the information they need quickly and accurately without stress.

[0975] The above is the specific processing flow of the system. With this system, users can ask a simple question and get a highly accurate answer that is best suited to their individual needs.

[0976] Example 2

[0977] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[0978] Conventional user support systems struggle to generate appropriate answers based on the user's individual needs. Furthermore, they are unable to generate answers that take the user's emotional state into account, which can result in a poor user experience. Furthermore, insufficient preprocessing of collected information results in suboptimal performance of machine learning models. Therefore, there is a need for a system that can provide emotionally sensitive answers while addressing the user's individual needs.

[0979] The identification process by the identification processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means. In this invention, the server includes a means for recognizing a user's emotion, a means for adjusting a response based on the user's emotion, a means for collecting personal information provided by the user, a means for integrating and collecting information from multiple account services, a means for preprocessing the collected data and inputting it into a machine learning model, a means for training the machine learning model using the preprocessed data, a means for generating an optimal answer to a question from the user, and a means for providing the generated answer to the user. This makes it possible to provide an optimal answer that takes into account the individual needs and emotional state of the user.

[0980] A "user" is an entity that inputs personal information and questions into the system.

[0981] A "server" is hardware or software that processes data collected from users, trains machine learning models, and generates and provides optimal answers.

[0982] A "terminal" is a device that provides an interface for a user to input personal information and questions, and communicates with a server.

[0983] "Personal Information" is personal data provided by a user, such as name, age, address, income, etc.

[0984] "Account services" are online services used by users, such as communication services, messaging services, and search services.

[0985] "Preprocessing" refers to the process of filtering out unnecessary information from collected data and normalizing the data.

[0986] A "machine learning model" is an algorithm that learns user needs and behavioral patterns based on collected and pre-processed data.

[0987] A "question" is an input regarding a question or request that a user poses to the system.

[0988] An "optimal answer" is a response to a user's question that is generated based on a machine learning model and takes into account the user's individual needs and emotional state.

[0989] An "emotion engine" is software that analyzes and recognizes the user's emotional state from their input.

[0990] "Filtering" is the process of removing unnecessary information from collected data.

[0991] "Normalization" is the process of standardizing and making consistent data formats.

[0992] "Tone" is the phrasing or mood of a response that reflects the user's emotional state.

[0993] The present invention is a system that collects and integrates personal information and information from multiple account services provided by users, inputs the preprocessed data into a machine learning model for learning, and further recognizes the user's emotions and appropriately uses the results to provide optimal answers to questions from users.

[0994] Data Collection Phase

[0995] The server first collects personal information provided by the user. This personal information includes name, age, address, and income. The device prompts the user to enter this information and sends it to the server. The server then aggregates and collects information from multiple account services (e.g., communication services, messaging services, search services, etc.). To obtain this information, it uses the APIs of each service to obtain the necessary user data. The device provides an interface for the user to enter API keys and authentication information.

[0996] Data preprocessing phase

[0997] The server pre-processes the collected personal and account information, which includes filtering unnecessary information and normalizing the data. This procedure improves the quality of the data and optimizes the performance of the learning model.

[0998] Learning Phase

[0999] The server uses the preprocessed data to train a machine learning model. This training process creates a model that reflects the individual needs and behavioral patterns of the user. The trained model is then stored on the server. For example, TensorFlow, a Python library, can be used for machine learning.

[1000] Emotion Recognition Phase

[1001] The server is equipped with an emotion engine that recognizes the user's emotions. When the user inputs a question, the device sends the input voice or text to the server, which analyzes it and recognizes the user's emotions. The emotion engine analyzes the nuances of the text and the tone of the voice to determine whether the user is excited, calm, or stressed.

[1002] Inquiry Processing Phase

[1003] When a user inputs a one-word question from their device, the device sends the question to the server, which uses the trained model to generate an optimal answer to the question. The generated answer is adjusted based on the user's individual profile as well as the user's emotions recognized by the emotion engine. This adjustment provides an answer that is most appropriate for the user's current mental state.

[1004] Finally, the terminal displays the answer received from the server to the user, and a special user interface allows the user to intuitively understand the answer and ask for more detailed information if necessary.

[1005] Specific examples

[1006] For example, consider a user entering a one-word question like, "I want a new refrigerator." If the emotion engine recognizes that the user is a little excited, the server will generate a response in a tone appropriate to that emotion. For example, the response might be something like, "Thank you for your interest. The perfect refrigerator for you is the YY model from XX. Also, please note that there is a discount available at your local recycling center." This provides clear and helpful information while moderating the user's excitement.

[1007] In this system, the user is given a prompt such as "I want a new refrigerator," and is instructed to "generate an appropriate response if the user is excited." In this way, the system provides an optimal answer that takes the user's emotions into account.

[1008] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1009] Step 1:

[1010] The terminal provides an interface that allows the user to input personal information (such as name, age, address, and income). The data to be input is personal information such as name, age, address, and income. The user inputs this information and sends the data from the terminal to a server. The server receives the input personal information and stores it in a database. This stores the user's basic information in the database.

[1011] Step 2:

[1012] The device provides an interface for the user to enter API keys and authentication information for each account service (communication service, messaging service, search service, etc.). The data entered is the API key and authentication information for each service. The user enters this information and sends it from the device to the server. The server uses the received API key and authentication information to call the API of each service and obtain the necessary user data. The obtained data is stored on the server, which allows information from multiple account services to be integrated.

[1013] Step 3:

[1014] The server preprocesses the collected personal and account information. The input for data preprocessing is the collected raw data, which includes unnecessary information. The server first filters out the unnecessary information to improve the quality of the input data. Next, it normalizes the remaining data and converts it into a consistent format. For example, standardizing the format of addresses ensures data integrity. The preprocessed data is then saved back into the database.

[1015] Step 4:

[1016] The server uses the preprocessed data to train a machine learning model. The input for this step is the high-quality data after preprocessing. The server trains the machine learning model using, for example, the Python library TensorFlow. The learning process incorporates user needs and behavioral patterns. The model after this training is completed is stored on the server. This results in a customized model that corresponds to specific user needs.

[1017] Step 5:

[1018] The terminal provides an interface for inputting questions from the user. The input data is the user's question in text or voice. The user inputs the question and sends it from the terminal to the server. The server passes the question to an emotion engine, which analyzes the nuances of the text or voice to determine the user's emotional state. For example, it analyzes information such as whether the user is excited, calm, or stressed.

[1019] Step 6:

[1020] The server uses the emotional information obtained from the emotion engine and the trained model to generate the optimal answer to the user's question. The input for this step is the user's question and emotional information. The server generates an answer in a tone that matches the user's current emotional state. The generated answer is adjusted based on the user's profile and emotion recognition. This prepares the optimal answer according to the emotion.

[1021] Step 7:

[1022] The server sends the generated optimal answer to the terminal. The terminal receives the answer sent from the server and displays it to the user. The output of this step is the optimal answer to the user's question. A special user interface allows the user to intuitively understand the answer and even request more detailed information. This allows the information the user is looking for to be provided quickly and appropriately.

[1023] (Application example 2)

[1024] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1025] In modern content delivery services, it is important to efficiently recommend content that meets users' needs. However, existing systems simply recommend content without considering the user's individual emotional state, which fails to sufficiently increase user satisfaction. Furthermore, they lack a mechanism for integrating data from multiple account services, analyzing user emotions, and proposing optimal content. Therefore, there is a need for a system that efficiently collects and analyzes users' personal information and information from multiple account services, and recommends content that best suits the user's emotions.

[1026] The identification process by the identification processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for collecting personal information provided by the user, means for integrating and collecting information from multiple account services, means for preprocessing the collected data and inputting it into a learning model, means for training a machine learning model using the preprocessed data, means for analyzing the user's emotions, and means for recommending content based on the results of the emotion analysis. This makes it possible to provide optimal content according to the user's emotional state.

[1027] "User" means an individual or a person associated with an individual who uses the system to provide information.

[1028] "Personal information" refers to data related to the user himself / herself, such as the user's name, age, address, hobbies, preferences, and income.

[1029] "Account service" refers to multiple online services to which a user is registered (e.g., social networking services, video streaming services, music streaming services, etc.).

[1030] A "machine learning model" refers to an algorithm or system that uses collected data to learn and make predictions.

[1031] "Emotion analysis means" refers to a system or algorithm that analyzes text or voice data entered by a user to determine their emotional state (e.g., positive, negative, neutral, etc.).

[1032] "Content" refers to digital content such as videos, music, and books, and indicates data and services provided to users.

[1033] "Preprocessing" refers to the process of filtering and normalizing collected data to make it suitable for machine learning models.

[1034] "API" refers to the application programming interface provided by each service, which allows other systems and applications to use the service's functions and data.

[1035] "Server" refers to a computer or network system that collects, processes, analyzes information from users, and returns the results.

[1036] "Recommendation" refers to the act of suggesting the most suitable content based on the user's past behavioral data and current emotional state.

[1037] The present invention is a system that collects and integrates personal information and information from multiple account services provided by users, inputs the preprocessed data into a machine learning model for learning, and then recognizes the user's emotions and appropriately uses the results to provide optimal answers to questions and requests from users.

[1038] System configuration and operation

[1039] Data Collection Phase

[1040] The server first collects personal information provided by the user, including name, age, address, hobbies, preferences, income, etc. The terminal prompts the user to enter this information and sends it to the server.

[1041] Next, the server aggregates and collects information from multiple account services (e.g., social networking sites, video streaming services, music streaming services, etc.). To obtain this information, it uses the APIs of each service to obtain the necessary user data. The device provides an interface for the user to enter their API key and authentication information.

[1042] Data preprocessing phase

[1043] The server pre-processes the collected personal and account information, which includes filtering unnecessary information and normalizing the data. This procedure improves the quality of the data and optimizes the performance of the learning model.

[1044] Learning Phase

[1045] The server uses the preprocessed data to train a machine learning model. This training process creates a model that reflects the individual needs and behavioral patterns of the user. Once trained, the model is stored on the server.

[1046] Emotion Recognition Phase

[1047] The server is equipped with an emotion engine that recognizes the user's emotions. When a user inputs a question or request, the device sends the input voice or text to the server, which analyzes it and recognizes the user's emotions. The emotion engine analyzes the nuances of the text and the tone of the voice to determine whether the user is excited, calm, or stressed.

[1048] Inquiry Processing Phase

[1049] When a user inputs a question or request from their device, the device sends the question or request to the server. The server uses the trained model to generate the optimal answer to the question. The generated answer is adjusted based on the individual user profile and the user's emotions recognized by the emotion engine. This adjustment provides an answer that is best suited to the user's current mental state.

[1050] Examples and prompts

[1051] For example, if a user inputs "I'm tired, please recommend some relaxing music," the emotion engine will recognize that the user is seeking relaxation. The server will then select content appropriate for that emotional state, such as healing music or a relaxing movie, and recommend it to the user in the form of "Thank you for your hard work. How about this relaxing music?"

[1052] Example prompt sentence:

[1053] "Username: Yamada Taro, Age: 30, Address: Shinjuku-ku, Tokyo. He asked for recommendations for new movies and music. He seems to be stressed at the moment."

[1054] This system will recommend optimal content based on the user's emotional state, providing a more personalized experience and significantly improving user satisfaction.

[1055] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1056] Step 1: Collect user information

[1057] Users use their devices to input and collect personal information such as their name, age, address, hobbies, and preferences. This information is sent to the server.

[1058] Input: Personal information such as name, age, address, hobbies, preferences, etc.

[1059] Output: Personal information data stored on the server

[1060] Specific operation: The terminal sends the input data, and the server stores it in the database.

[1061] Step 2: Integrated collection of account information

[1062] The user enters an API key and authentication information into their device to obtain information such as playlists and viewing history, and sends that information to the server. The server then obtains the necessary data from multiple account services (e.g., social networking sites, video streaming services) via the API.

[1063] Input: API key or authentication information

[1064] Output: User data obtained from the account service

[1065] Specific operation: The device sends the user's authentication information, and the server calls various APIs to collect data.

[1066] Step 3: Preprocessing the data

[1067] The server pre-processes the collected personal and account information, which includes filtering unnecessary information and normalizing the data.

[1068] Input: Collected user personal and account information

[1069] Output: Preprocessed data

[1070] Specific operation: The server performs data cleansing and normalization of the data.

[1071] Step 4: Training the machine learning model

[1072] The server uses the preprocessed data to train a machine learning model, which is then adjusted based on the user's behavioral patterns and preferences and stored on the server.

[1073] Input: Preprocessed data

[1074] Output: A trained machine learning model

[1075] What it does: The server splits the data into training sets and applies machine learning algorithms to build models.

[1076] Step 5: Emotion Recognition

[1077] The user inputs a question or request into the device, which then sends the input voice or text to the server, where the emotion engine analyzes it to recognize the user's emotions.

[1078] Input: Voice or text of the user's question or request

[1079] Output: Perceived emotional state (e.g., positive, negative, neutral)

[1080] Specific operation: The server uses an emotion analysis algorithm to analyze the user's text and voice data and determine their emotional state.

[1081] Step 6: Recommending the best content

[1082] The server uses the results of the trained machine learning model and emotion engine to recommend the most suitable content to the user, and the recommended content is tailored based on the user's current emotional state.

[1083] Input: Perceived emotional state and user profile

[1084] Output: Recommended content

[1085] Specific operation: The server uses a machine learning model to select the optimal content based on user data and emotional data, and sends the results to the device.

[1086] Step 7: Provide to users

[1087] The terminal displays the most suitable content provided by the server to the user, who can then view the recommended content and request more detailed information.

[1088] Input: Recommended content provided by the server

[1089] Output: Content information displayed on the device

[1090] Specific operation: The device uses a user interface to display the recommended content and allow the user to access it.

[1091] The specific processing unit 290 transmits the result of the specific processing to the headset type terminal 314. In the headset type terminal 314, the control unit 46A causes the speaker 240 and the display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1092] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1093] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the headset type terminal 314.

[1094] [Fourth embodiment]

[1095] FIG. 7 shows an example of the configuration of a data processing system 410 according to the fourth embodiment.

[1096] 7, a data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[1097] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1098] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a control target 443. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the control target 443 are also connected to the bus 52.

[1099] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1100] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1101] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1102] The control object 443 includes a display device, LEDs in the eyes, and motors for driving the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the emotions of the robot 414 can be expressed by controlling these motors. In addition, the facial expressions of the robot 414 can also be expressed by controlling the light emission state of the LEDs in the eyes of the robot 414.

[1103] Fig. 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Fig. 8, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1104] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1105] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1106] In the robot 414, the processor 46 performs the reception output process. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1107] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1108] The present invention is a system that collects and integrates personal information and information from multiple account services provided by users, inputs the preprocessed data into a machine learning model, and trains it to provide optimal answers to questions from users.

[1109] System configuration and operation

[1110] Data Collection Phase

[1111] The server first collects personal information provided by the user, including name, age, address, income, etc. The terminal prompts the user to enter this information and sends it to the server.

[1112] The server then aggregates and collects information from multiple account services (e.g., communication services, messaging services, search services, etc.). To obtain this information, it uses the APIs of each service to obtain the necessary user data. The device provides an interface for the user to enter their API key and authentication information.

[1113] Data preprocessing phase

[1114] The server preprocesses the collected data and converts it into a format suitable for the learning model. Preprocessing includes filtering unnecessary information and normalizing the data. This procedure improves the quality of the data and optimizes the performance of the learning model.

[1115] Learning Phase

[1116] The server uses the preprocessed data to train a machine learning model, which then creates a model that reflects the individual needs and behavioral patterns of the user. The server then stores the trained model and uses it for subsequent query processing.

[1117] Inquiry Processing Phase

[1118] When a user types a one-word question into their device, the device sends the question to the server, which uses the trained model to generate the best answer for the question. The generated answer is customized based on the individual user profile, so it can meet the user's needs with high accuracy.

[1119] Finally, the terminal displays the answer received from the server to the user, and a special user interface allows the user to intuitively understand the answer and ask for more detailed information if necessary.

[1120] Specific examples

[1121] For example, consider a case where a user is looking for a new refrigerator. The user enters a one-word question such as, "I want a new refrigerator." When the device sends this question to the server, the server will suggest the optimal refrigerator model based on the user's financial situation and living environment.

[1122] The server selects the refrigerator with the best price and energy efficiency, taking into account the user's income and the electricity rates in the area where they live, for example. It also provides information on local recycling services and discount campaigns. In this way, users can obtain information on the refrigerator that is best suited to them by simply asking a question.

[1123] The above is an embodiment of the present invention, and the specific configuration and operation of the system have been described. This system allows users to easily obtain optimal information, thereby providing high convenience.

[1124] The processing flow will be explained below.

[1125] Step 1:

[1126] The user enters personal information, including name, age, address, income, etc. The entered information is collected by the terminal and sent to the server.

[1127] Step 2:

[1128] The server aggregates and collects information from multiple account services (e.g., communication services, messaging services, search services). The device provides an interface for users to enter API keys and authentication information. The server obtains user data using the APIs of each service.

[1129] Step 3:

[1130] The server preprocesses the collected personal and account information, which includes filtering out unnecessary information and normalizing the data, converting it into a format suitable for the learning model.

[1131] Step 4:

[1132] The server uses the preprocessed data to train a machine learning model. This training process creates a model that reflects the individual needs and behavioral patterns of the user. Once trained, the model is stored on the server.

[1133] Step 5:

[1134] The user inputs a one-word question into the terminal, for example, "I want a new refrigerator." The terminal then sends this question to the server.

[1135] Step 6:

[1136] The server uses the trained model to generate optimal answers to user questions, and suggests optimal products and services based on the user's profile and the content of the question.

[1137] Step 7:

[1138] The terminal displays the answer received from the server to the user, which may include, for example, recommended refrigerator models and prices, local recycling services, and discount campaign information.

[1139] The above is the specific processing flow of the system program. This allows users to easily obtain the most appropriate information, providing a high level of convenience.

[1140] Example 1

[1141] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1142] In conventional systems, when integrating and using information provided by users with information from multiple account services, it has been difficult to ensure data accuracy and consistency, and to provide optimal information to individual users. Furthermore, there has been a need to efficiently train machine learning models and generate fast, highly accurate answers to user questions. The present invention aims to solve these problems and provide users with highly accurate, optimal information.

[1143] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1144] In this invention, the server includes: means for collecting personal information provided by a user; means for integrating and collecting information from multiple account services; means for preprocessing the collected data and inputting it into a learning model; means for training a machine learning model using the preprocessed data; means for generating optimal answers to questions from users; means for a terminal to acquire authentication information from the user and send it to the server; means for the server to acquire user data using APIs of each service; means for the server to normalize data and filter unnecessary data; means for training the preprocessed data using a machine learning algorithm and saving the model; and means for accepting user questions, generating answers using the saved model, and sending them to the terminal. This makes it possible to provide users with quick and accurate answers that meet their individual needs.

[1145] A "user" is an individual or group that uses the system and provides personal information, authentication information, etc.

[1146] A "server" is a computer system that collects, stores, and processes information provided by users and trains and executes machine learning models.

[1147] A "terminal" is a device that a user uses to enter information or submit a question, and is an interface that communicates with a server.

[1148] "Personal information" refers to basic information about a user, including data such as name, age, address, and income.

[1149] "Account services" refer to various online services used by users, including communication services, messaging services, search services, and the like.

[1150] "API" stands for Application Program Interface, a set of rules and protocols for exchanging data between different software programs.

[1151] "Data normalization" is the process of converting collected data into a consistent format, a process undertaken to improve data quality.

[1152] "Data filtering" is the process of removing information from a dataset that is not useful to a machine learning model.

[1153] A "machine learning model" is an algorithm or system that uses collected and preprocessed data to learn and perform a specific task.

[1154] A "learning algorithm" is a mathematical or statistical method for using data to train a machine learning model and improve its performance.

[1155] "Authentication information" refers to information required for a user to access each account service, including API keys and passwords.

[1156] A "question" is an inquiry that a user inputs into the system, and is information that triggers the server to generate the most appropriate answer.

[1157] An "answer" is a response generated by the server to a user's question, customized based on the user's profile and data.

[1158] MODE FOR CARRYING OUT THE INVENTION

[1159] The present invention is a system that collects and integrates personal information and information from multiple account services provided by users, inputs the preprocessed data into a machine learning model, and trains it to provide optimal answers to questions from users.

[1160] System Configuration

[1161] The system mainly consists of a server, a terminal, and a user. Each component is explained in detail below.

[1162] Data Collection Phase

[1163] The user uses the device to enter personal information, such as name, age, address, and income. The entered information is sent to the server via the device. The server then retrieves the user data using APIs of multiple account services (e.g., communication services, messaging services, search services, etc.). The device provides an interface for the user to enter the necessary API keys and authentication information. Protocols such as REST API and OAuth are used to collect this type of data.

[1164] Data preprocessing phase

[1165] The server preprocesses the collected data, converting it into a format suitable for the learning model. Preprocessing includes filtering out unnecessary information and normalizing the data. For example, it converts address information into a standard format and standardizes the currency unit of income information. Techniques used include regular expressions and statistical methods.

[1166] Learning Phase

[1167] The server uses the preprocessed data to train a machine learning model. For training, a machine learning framework such as TensorFlow or PyTorch is used. This creates a model that reflects the individual needs and behavioral patterns of the user. The trained model is saved in the server's file system or database. Common save formats include model.h5 and model.pt.

[1168] Inquiry Processing Phase

[1169] When a user inputs a question from their device, the device sends the question to the server, which uses the trained model to generate the optimal answer to the question. For example, if a user asks, "I want a new refrigerator," the server will suggest the optimal refrigerator model based on the user's financial situation and living environment. The answer is sent to the device and displayed to the user.

[1170] Specific examples

[1171] For example, consider a case where a user inputs "I want a new refrigerator." When the device sends this question to the server, the server will suggest the optimal refrigerator model based on the user's personal information and collected account service data. Taking into account income and local electricity rates, the server will select the refrigerator with the best price range and energy efficiency, and also provide information on local recycling services and discount campaigns.

[1172] Here is an example prompt:

[1173] User: I want a new refrigerator.

[1174] Terminal: Sends user questions to the server.

[1175] Server: Recommends the optimal refrigerator model based on the user's financial situation and living environment. For example, it selects the refrigerator with the best price range and energy efficiency, taking into account income and local electricity rates, and also provides information on local recycling services and discount campaigns.

[1176] Terminal: Shows the suggested refrigerator information to the user.

[1177] This system allows users to easily obtain the most appropriate information, providing a high level of convenience.

[1178] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1179] System processing steps

[1180] Step 1: Collect user information

[1181] Input: The user enters personal information into the device.

[1182] Specific behavior: The user adds personal information such as name, age, address, income, etc. to the input form. Once the input is complete, the user clicks the "Submit" button.

[1183] Data processing: The terminal sends the entered personal information to the server via the REST API.

[1184] Output: The server receives the user's personal information and stores it in a database.

[1185] Step 2: Integrate your account service information

[1186] Input: The user enters the authentication information for each account service into the device.

[1187] Specific operation: The user enters the API key and authentication information into the input form and clicks the "Submit" button.

[1188] Data processing: The terminal sends authentication information to the server via REST API, and the server retrieves user data using the API of each service.

[1189] Output: The server retrieves user data from each account service and integrates it into a database.

[1190] Step 3: Normalize the data

[1191] Input: Personal information and account service information collected by the server

[1192] Specific operation: The server converts the address information into a unified format and unifies the currency unit of the income information.

[1193] Data manipulation: Using regular expressions and statistical techniques to transform data into a consistent format.

[1194] Output: The server generates the normalized data and stores it in a database.

[1195] Step 4: Filtering out unnecessary data

[1196] Input: Server-normalized data

[1197] Specific operation: The server filters out unnecessary information (e.g., phone numbers, email addresses).

[1198] Data processing: The server uses scripts to automatically remove unnecessary fields.

[1199] Output: The server generates the filtered data and stores it in a database.

[1200] Step 5: Prepare the training data

[1201] Input: Server filtered data

[1202] What it does: The server converts the data into a format suitable for the machine learning model.

[1203] Data processing: The preprocessed data is formatted and converted into a format that is easy for machine learning algorithms to use (e.g., CSV or JSON).

[1204] Output: The server generates a dataset that can be trained on.

[1205] Step 6: Training the model

[1206] Input: Training data prepared by the server

[1207] Specific operation: The server trains machine learning models using TensorFlow or PyTorch.

[1208] Data processing: The server inputs the training data into the algorithm to train the model.

[1209] Output: The server generates the trained model and saves it to the file system (in the format model.h5 or model.pt).

[1210] Step 7: Ask questions

[1211] Input: The user types a question into the terminal

[1212] Specific behavior: The user enters a question into the input form and clicks the "Submit" button.

[1213] Data processing: The terminal sends the question to the server via the REST API.

[1214] Output: The server receives the query and records it in a database for processing.

[1215] Step 8: Generate an answer

[1216] Input: The question received by the server and the trained model

[1217] What it does: The server uses the trained model to analyze the question and generate the best answer.

[1218] Data Calculation: The server analyzes the question and generates an answer based on the user profile and model.

[1219] Output: The server records the generated answer in a database and sends it to the terminal.

[1220] Step 9: View your answers

[1221] Input: The answer sent by the server

[1222] Specific operation: The terminal displays the answer received from the server to the user.

[1223] Data processing: The terminal embeds the response data into a template and converts it into a format for display.

[1224] Output: The user can check the answer on the device.

[1225] The above are the specific processing steps of this system, which allows users to easily obtain optimal information quickly and accurately.

[1226] (Application example 1)

[1227] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1228] Today, many users use multiple electronic payment services, and checking the benefits and campaign information for each service individually is a significant burden. It is also difficult to provide accurate savings advice based on the user's consumption patterns. As a result, many users miss out on appropriate savings methods and useful campaign information. Therefore, there is a need for a system that can integrate and collect information from users' personal information and multiple account services, and provide optimal savings advice and campaign information based on each user's individual consumption patterns.

[1229] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1230] In this invention, the server includes means for collecting personal information provided by a user, means for integrating and collecting information from multiple account services, means for preprocessing the collected data and inputting it into a learning model, means for training a machine learning model using the preprocessed data, means for analyzing the user's consumption pattern and generating optimal saving methods and campaign information, and means for providing the generated saving methods and campaign information to the user. This enables the user to integrate and analyze data from multiple information sources and easily obtain accurate saving advice and useful campaign information based on their own consumption pattern.

[1231] "User" refers to an individual or corporation that uses the system.

[1232] "Personal Information" refers to information that can be used to identify a specific individual, such as a user's name, age, address, or income.

[1233] "Account Service" refers to a service that provides information related to accounts created by users with electronic payment services and other online services.

[1234] "Means for integrating and collecting information" refers to the process of centralizing and collecting user data from multiple account services.

[1235] "Preprocessing" refers to the process of filtering unnecessary information from collected data and normalizing the data.

[1236] "Means for inputting to a learning model" refers to a method for feeding preprocessed data to a machine learning algorithm for learning.

[1237] A "machine learning model" refers to an algorithm that uses large amounts of data to learn specific patterns and relationships and then makes predictions and classifications for new data.

[1238] "Consumption patterns" refers to information obtained by identifying specific spending tendencies or behavioral patterns from a user's past spending history.

[1239] "Savings tips" refer to specific measures or techniques suggested to help users reduce their spending.

[1240] "Campaign Information" refers to information regarding discounts and benefits offered by electronic payment services and related services.

[1241] "Means of generation" refers to the process of using a machine learning model to create optimal savings methods and campaign information based on a user's individual information.

[1242] The "means of providing" refers to a method of communicating the generated savings methods and campaign information to users.

[1243] The present invention is a system that integrates personal information collected from users and information from multiple account services, and uses the preprocessed data to train a machine learning model. The method by which this system provides optimal answers to user questions is described in detail below.

[1244] System configuration and operation

[1245] Data Collection Phase

[1246] The server first collects personal information provided by the user, including name, age, address, income, etc. The terminal prompts the user to enter this information and sends it to the server.

[1247] The server then aggregates and collects information from multiple account services (such as electronic payment services and other online services). To obtain this information, it uses the API of each service to obtain the necessary user data. The device provides an interface for the user to enter their API key and authentication information.

[1248] Data preprocessing phase

[1249] The server preprocesses the collected data and converts it into a format suitable for the learning model. Preprocessing includes filtering unnecessary information and normalizing the data. This procedure improves the quality of the data and optimizes the performance of the learning model.

[1250] Learning Phase

[1251] The server uses the preprocessed data to train a machine learning model, which then creates a model that reflects the individual user's consumption patterns. The server then stores the trained model and uses it for subsequent query processing.

[1252] Inquiry Processing Phase

[1253] When a user inputs a question about saving money or campaign information on their device, the device sends the question to the server, which uses the trained model to generate the optimal answer to the question. The generated answer is customized based on the individual user profile, so it can meet the user's needs with high accuracy.

[1254] Finally, the terminal displays the answer received from the server to the user, and a special user interface allows the user to intuitively understand the answer and ask for more detailed information if necessary.

[1255] Specific examples

[1256] For example, consider a case where a user wants to know about their future spending patterns. The user inputs the question, "Please tell me about your future spending patterns." When the device sends this question to the server, the server will suggest optimal savings methods and useful campaign information based on the user's income and past spending data.

[1257] Specific prompt examples:

[1258] Prompt: Tell us about your future spending patterns

[1259] Response generation context:

[1260] Username: Taro

[1261] Age: 30

[1262] Income: 500,000 yen per month

[1263] Historical spending data: [Provided in list format]

[1264] Generated response:

[1265] Basic money saving advice

[1266] Electronic payment service benefits and campaign information

[1267] In this way, the system of the present invention provides useful information to the user and achieves high convenience.

[1268] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1269] Step 1:

[1270] Data collection

[1271] The server first collects personal information (such as name, age, address, and income) from the user via the device. This input data is sent to the server in JSON format.

[1272] Next, the server obtains the user's transaction history and account information through the APIs of multiple electronic payment services, and the user enters the API key and authentication information on the terminal.

[1273] The server integrates the acquired data and stores it as a single dataset. The input is user data from each service, and the output is the integrated dataset.

[1274] Step 2:

[1275] Data Preprocessing

[1276] The server then filters unnecessary information from the combined data set and extracts only the information needed, such as past transaction history and income data.

[1277] The extracted data is then normalized, which improves the quality of the data and allows the learning model to analyze it more efficiently.

[1278] The preprocessed data is converted into a format suitable for the machine learning model. The input is the unfiltered dataset, and the output is the preprocessed dataset.

[1279] Step 3:

[1280] Training a machine learning model

[1281] The server uses the preprocessed data to train a machine learning model, using algorithms such as Linear Regression and Random Forest.

[1282] The training process creates a model that reflects your income and spending patterns. The input is the preprocessed dataset, and the output is the trained model.

[1283] The trained model is stored on the server and used for subsequent query processing.

[1284] Step 4:

[1285] Query Processing

[1286] The user inputs a question into the terminal. For example, a simple question such as "Please tell me about your future spending patterns" is input.

[1287] The terminal sends a question from the user to the server.

[1288] Step 5:

[1289] Generating optimal answers

[1290] The server uses the trained model to generate the best answer to the user's question. Based on this prompt, the model generates an answer based on the user's consumption patterns.

[1291] The generated answers are customized for each user and include specific savings methods and campaign information. The input is the user's question and the trained model, and the output is the optimal answer.

[1292] Step 6:

[1293] Providing answers

[1294] The server sends the generated response to the terminal.

[1295] The terminal displays the answer received from the server to the user, who can then consider specific actions based on the presented information. The input is the optimal answer, and the output is the displayed information.

[1296] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1297] The present invention is a system that collects and integrates personal information and information from multiple account services provided by users, inputs the preprocessed data into a machine learning model for learning, and further recognizes the user's emotions and appropriately uses the results to provide optimal answers to questions from users.

[1298] System configuration and operation

[1299] Data Collection Phase

[1300] The server first collects personal information provided by the user, such as name, age, address, and income, and the terminal prompts the user to enter this information and sends it to the server.

[1301] The server then aggregates and collects information from multiple account services (e.g., communication services, messaging services, search services, etc.). To obtain this information, it uses the APIs of each service to obtain the necessary user data. The device provides an interface for the user to enter their API key and authentication information.

[1302] Data preprocessing phase

[1303] The server pre-processes the collected personal and account information, which includes filtering unnecessary information and normalizing the data. This procedure improves the quality of the data and optimizes the performance of the learning model.

[1304] Learning Phase

[1305] The server uses the preprocessed data to train a machine learning model. This training process creates a model that reflects the individual needs and behavioral patterns of the user. Once the model is trained, it is stored on the server.

[1306] Emotion Recognition Phase

[1307] The server is equipped with an emotion engine that recognizes the user's emotions. When the user inputs a question, the device sends the input voice or text to the server, which analyzes it and recognizes the user's emotions. The emotion engine analyzes the nuances of the text and the tone of the voice to determine whether the user is excited, calm, or stressed.

[1308] Inquiry Processing Phase

[1309] When a user inputs a one-word question from their device, the device sends the question to the server, which uses the trained model to generate an optimal answer to the question. The generated answer is adjusted based on the user's individual profile as well as the user's emotions recognized by the emotion engine. This adjustment provides an answer that is most appropriate for the user's current mental state.

[1310] Finally, the terminal displays the answer received from the server to the user, and a special user interface allows the user to intuitively understand the answer and ask for more detailed information if necessary.

[1311] Specific examples

[1312] For example, imagine a user is looking for a new refrigerator. The user enters a one-word question: "I want a new refrigerator." If the emotion engine detects that the user is a little excited, the server generates a response in a tone appropriate to that emotion. For example, the response might be something like, "Thank you for your interest. The perfect refrigerator for you is the YY model from XX. Also, please note that there is a discount available at your local recycling center." This provides clear and helpful information while moderating the user's excitement.

[1313] In this way, by combining emotion recognition functionality, the present invention makes it possible to provide information tailored to individual user needs with even greater precision. This allows users to obtain the information most suited to them without stress, simply by asking a simple question, and enjoys high convenience.

[1314] The processing flow will be explained below.

[1315] Step 1:

[1316] The user enters personal information, including name, age, address, income, etc. The entered information is collected by the terminal and sent to the server.

[1317] Step 2:

[1318] The server collects and integrates information from multiple account services (e.g., communication services, messaging services, search services). The device provides an interface for the user to enter API keys and authentication information, and the server obtains user data using the APIs of each service.

[1319] Step 3:

[1320] The server preprocesses the collected personal and account information. This preprocessing includes filtering out unnecessary information and normalizing the data. This preprocessing improves the quality of the data and converts it into a format suitable for the learning model.

[1321] Step 4:

[1322] The server uses the preprocessed data to train a machine learning model. This training process creates a model that reflects the individual needs and behavioral patterns of the user. Once trained, the model is stored on the server.

[1323] Step 5:

[1324] The user inputs a question into the terminal. For example, the user inputs a question such as "I want a new refrigerator." The terminal then transmits the input text or voice data to the server.

[1325] Step 6:

[1326] The server uses an emotion engine to analyze the user's input text and voice to recognize the user's emotions. The emotion engine analyzes, for example, the nuances of the text and the tone of the voice to determine whether the user is excited, calm, or stressed.

[1327] Step 7:

[1328] The server uses the trained model to generate the best answer to the user's question, and the emotion engine adjusts the answer based on the user's emotions. For example, if the user is excited, the server creates an answer with a calming tone.

[1329] Step 8:

[1330] The server sends the generated answer to the terminal, which displays the answer to the user. Specifically, the answer includes information about the suggested products and special offers.

[1331] Step 9:

[1332] The user checks the answers displayed on the device and requests more detailed information if necessary. This process allows the user to obtain the information they need quickly and accurately without stress.

[1333] The above is the specific processing flow of the system. With this system, users can ask a simple question and get a highly accurate answer that is best suited to their individual needs.

[1334] Example 2

[1335] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1336] Conventional user support systems struggle to generate appropriate answers based on the user's individual needs. Furthermore, they are unable to generate answers that take the user's emotional state into account, which can result in a poor user experience. Furthermore, insufficient preprocessing of collected information results in suboptimal performance of machine learning models. Therefore, there is a need for a system that can provide emotionally sensitive answers while addressing the user's individual needs.

[1337] The identification process by the identification processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means. In this invention, the server includes a means for recognizing a user's emotion, a means for adjusting a response based on the user's emotion, a means for collecting personal information provided by the user, a means for integrating and collecting information from multiple account services, a means for preprocessing the collected data and inputting it into a machine learning model, a means for training the machine learning model using the preprocessed data, a means for generating an optimal answer to a question from the user, and a means for providing the generated answer to the user. This makes it possible to provide an optimal answer that takes into account the individual needs and emotional state of the user.

[1338] A "user" is an entity that inputs personal information and questions into the system.

[1339] A "server" is hardware or software that processes data collected from users, trains machine learning models, and generates and provides optimal answers.

[1340] A "terminal" is a device that provides an interface for a user to input personal information and questions, and communicates with a server.

[1341] "Personal Information" is personal data provided by a user, such as name, age, address, income, etc.

[1342] "Account services" are online services used by users, such as communication services, messaging services, and search services.

[1343] "Preprocessing" refers to the process of filtering out unnecessary information from collected data and normalizing the data.

[1344] A "machine learning model" is an algorithm that learns user needs and behavioral patterns based on collected and pre-processed data.

[1345] A "question" is an input regarding a question or request that a user poses to the system.

[1346] An "optimal answer" is a response to a user's question that is generated based on a machine learning model and takes into account the user's individual needs and emotional state.

[1347] An "emotion engine" is software that analyzes and recognizes the user's emotional state from their input.

[1348] "Filtering" is the process of removing unnecessary information from collected data.

[1349] "Normalization" is the process of standardizing and making consistent data formats.

[1350] "Tone" is the phrasing or mood of a response that reflects the user's emotional state.

[1351] The present invention is a system that collects and integrates personal information and information from multiple account services provided by users, inputs the preprocessed data into a machine learning model for learning, and further recognizes the user's emotions and appropriately uses the results to provide optimal answers to questions from users.

[1352] Data Collection Phase

[1353] The server first collects personal information provided by the user. This personal information includes name, age, address, and income. The device prompts the user to enter this information and sends it to the server. The server then aggregates and collects information from multiple account services (e.g., communication services, messaging services, search services, etc.). To obtain this information, it uses the APIs of each service to obtain the necessary user data. The device provides an interface for the user to enter API keys and authentication information.

[1354] Data preprocessing phase

[1355] The server pre-processes the collected personal and account information, which includes filtering unnecessary information and normalizing the data. This procedure improves the quality of the data and optimizes the performance of the learning model.

[1356] Learning Phase

[1357] The server uses the preprocessed data to train a machine learning model. This training process creates a model that reflects the individual needs and behavioral patterns of the user. The trained model is then stored on the server. For example, TensorFlow, a Python library, can be used for machine learning.

[1358] Emotion Recognition Phase

[1359] The server is equipped with an emotion engine that recognizes the user's emotions. When the user inputs a question, the device sends the input voice or text to the server, which analyzes it and recognizes the user's emotions. The emotion engine analyzes the nuances of the text and the tone of the voice to determine whether the user is excited, calm, or stressed.

[1360] Inquiry Processing Phase

[1361] When a user inputs a one-word question from their device, the device sends the question to the server, which uses the trained model to generate an optimal answer to the question. The generated answer is adjusted based on the user's individual profile as well as the user's emotions recognized by the emotion engine. This adjustment provides an answer that is most appropriate for the user's current mental state.

[1362] Finally, the terminal displays the answer received from the server to the user, and a special user interface allows the user to intuitively understand the answer and ask for more detailed information if necessary.

[1363] Specific examples

[1364] For example, consider a user entering a one-word question like, "I want a new refrigerator." If the emotion engine recognizes that the user is a little excited, the server will generate a response in a tone appropriate to that emotion. For example, the response might be something like, "Thank you for your interest. The perfect refrigerator for you is the YY model from XX. Also, please note that there is a discount available at your local recycling center." This provides clear and helpful information while moderating the user's excitement.

[1365] In this system, the user is given a prompt such as "I want a new refrigerator," and is instructed to "generate an appropriate response if the user is excited." In this way, the system provides an optimal answer that takes the user's emotions into account.

[1366] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1367] Step 1:

[1368] The terminal provides an interface that allows the user to input personal information (such as name, age, address, and income). The data to be input is personal information such as name, age, address, and income. The user inputs this information and sends the data from the terminal to a server. The server receives the input personal information and stores it in a database. This stores the user's basic information in the database.

[1369] Step 2:

[1370] The device provides an interface for the user to enter API keys and authentication information for each account service (communication service, messaging service, search service, etc.). The data entered is the API key and authentication information for each service. The user enters this information and sends it from the device to the server. The server uses the received API key and authentication information to call the API of each service and obtain the necessary user data. The obtained data is stored on the server, which allows information from multiple account services to be integrated.

[1371] Step 3:

[1372] The server preprocesses the collected personal and account information. The input for data preprocessing is the collected raw data, which includes unnecessary information. The server first filters out the unnecessary information to improve the quality of the input data. Next, it normalizes the remaining data and converts it into a consistent format. For example, standardizing the format of addresses ensures data integrity. The preprocessed data is then saved back into the database.

[1373] Step 4:

[1374] The server uses the preprocessed data to train a machine learning model. The input for this step is the high-quality data after preprocessing. The server trains the machine learning model using, for example, the Python library TensorFlow. The learning process incorporates user needs and behavioral patterns. The model after this training is completed is stored on the server. This results in a customized model that corresponds to specific user needs.

[1375] Step 5:

[1376] The terminal provides an interface for inputting questions from the user. The input data is the user's question in text or voice. The user inputs the question and sends it from the terminal to the server. The server passes the question to an emotion engine, which analyzes the nuances of the text or voice to determine the user's emotional state. For example, it analyzes information such as whether the user is excited, calm, or stressed.

[1377] Step 6:

[1378] The server uses the emotional information obtained from the emotion engine and the trained model to generate the optimal answer to the user's question. The input for this step is the user's question and emotional information. The server generates an answer in a tone that matches the user's current emotional state. The generated answer is adjusted based on the user's profile and emotion recognition. This prepares the optimal answer according to the emotion.

[1379] Step 7:

[1380] The server sends the generated optimal answer to the terminal. The terminal receives the answer sent from the server and displays it to the user. The output of this step is the optimal answer to the user's question. A special user interface allows the user to intuitively understand the answer and even request more detailed information. This allows the information the user is looking for to be provided quickly and appropriately.

[1381] (Application example 2)

[1382] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1383] In modern content delivery services, it is important to efficiently recommend content that meets users' needs. However, existing systems simply recommend content without considering the user's individual emotional state, which fails to sufficiently increase user satisfaction. Furthermore, they lack a mechanism for integrating data from multiple account services, analyzing user emotions, and proposing optimal content. Therefore, there is a need for a system that efficiently collects and analyzes users' personal information and information from multiple account services, and recommends content that best suits the user's emotions.

[1384] The identification process by the identification processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for collecting personal information provided by the user, means for integrating and collecting information from multiple account services, means for preprocessing the collected data and inputting it into a learning model, means for training a machine learning model using the preprocessed data, means for analyzing the user's emotions, and means for recommending content based on the results of the emotion analysis. This makes it possible to provide optimal content according to the user's emotional state.

[1385] "User" means an individual or a person associated with an individual who uses the system to provide information.

[1386] "Personal information" refers to data related to the user himself / herself, such as the user's name, age, address, hobbies, preferences, and income.

[1387] "Account service" refers to multiple online services to which a user is registered (e.g., social networking services, video streaming services, music streaming services, etc.).

[1388] A "machine learning model" refers to an algorithm or system that uses collected data to learn and make predictions.

[1389] "Emotion analysis means" refers to a system or algorithm that analyzes text or voice data entered by a user to determine their emotional state (e.g., positive, negative, neutral, etc.).

[1390] "Content" refers to digital content such as videos, music, and books, and indicates data and services provided to users.

[1391] "Preprocessing" refers to the process of filtering and normalizing collected data to make it suitable for machine learning models.

[1392] "API" refers to the application programming interface provided by each service, which allows other systems and applications to use the service's functions and data.

[1393] "Server" refers to a computer or network system that collects, processes, analyzes information from users, and returns the results.

[1394] "Recommendation" refers to the act of suggesting the most suitable content based on the user's past behavioral data and current emotional state.

[1395] The present invention is a system that collects and integrates personal information and information from multiple account services provided by users, inputs the preprocessed data into a machine learning model for learning, and then recognizes the user's emotions and appropriately uses the results to provide optimal answers to questions and requests from users.

[1396] System configuration and operation

[1397] Data Collection Phase

[1398] The server first collects personal information provided by the user, including name, age, address, hobbies, preferences, income, etc. The terminal prompts the user to enter this information and sends it to the server.

[1399] Next, the server aggregates and collects information from multiple account services (e.g., social networking sites, video streaming services, music streaming services, etc.). To obtain this information, it uses the APIs of each service to obtain the necessary user data. The device provides an interface for the user to enter their API key and authentication information.

[1400] Data preprocessing phase

[1401] The server pre-processes the collected personal and account information, which includes filtering unnecessary information and normalizing the data. This procedure improves the quality of the data and optimizes the performance of the learning model.

[1402] Learning Phase

[1403] The server uses the preprocessed data to train a machine learning model. This training process creates a model that reflects the individual needs and behavioral patterns of the user. Once the model is trained, it is stored on the server.

[1404] Emotion Recognition Phase

[1405] The server is equipped with an emotion engine that recognizes the user's emotions. When a user inputs a question or request, the device sends the input voice or text to the server, which analyzes it and recognizes the user's emotions. The emotion engine analyzes the nuances of the text and the tone of the voice to determine whether the user is excited, calm, or stressed.

[1406] Inquiry Processing Phase

[1407] When a user inputs a question or request from their device, the device sends the question or request to the server. The server uses the trained model to generate the optimal answer to the question. The generated answer is adjusted based on the individual user profile and the user's emotions recognized by the emotion engine. This adjustment provides an answer that is best suited to the user's current mental state.

[1408] Examples and prompts

[1409] For example, if a user inputs "I'm tired, please recommend some relaxing music," the emotion engine will recognize that the user is seeking relaxation. The server will then select content appropriate for that emotional state, such as healing music or a relaxing movie, and recommend it to the user in the form of "Thank you for your hard work. How about this relaxing music?"

[1410] Example prompt sentence:

[1411] "Username: Yamada Taro, Age: 30, Address: Shinjuku-ku, Tokyo. He asked for recommendations for new movies and music. He seems to be stressed at the moment."

[1412] This system will recommend optimal content based on the user's emotional state, providing a more personalized experience and significantly improving user satisfaction.

[1413] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1414] Step 1: Collect user information

[1415] Users use their devices to input and collect personal information such as their name, age, address, hobbies, and preferences. This information is sent to the server.

[1416] Input: Personal information such as name, age, address, hobbies, preferences, etc.

[1417] Output: Personal information data stored on the server

[1418] Specific operation: The terminal sends the input data, and the server stores it in the database.

[1419] Step 2: Integrated collection of account information

[1420] The user enters an API key and authentication information into their device to obtain information such as playlists and viewing history, and sends that information to the server. The server then obtains the necessary data from multiple account services (e.g., social networking sites, video streaming services) via the API.

[1421] Input: API key or authentication information

[1422] Output: User data obtained from the account service

[1423] Specific operation: The device sends the user's authentication information, and the server calls various APIs to collect data.

[1424] Step 3: Preprocessing the data

[1425] The server pre-processes the collected personal and account information, which includes filtering unnecessary information and normalizing the data.

[1426] Input: Collected user personal and account information

[1427] Output: Preprocessed data

[1428] Specific operation: The server performs data cleansing and normalization of the data.

[1429] Step 4: Training the machine learning model

[1430] The server uses the preprocessed data to train a machine learning model, which is then adjusted based on the user's behavioral patterns and preferences and stored on the server.

[1431] Input: Preprocessed data

[1432] Output: A trained machine learning model

[1433] What it does: The server splits the data into training sets and applies machine learning algorithms to build models.

[1434] Step 5: Emotion Recognition

[1435] The user inputs a question or request into the device, which then sends the input voice or text to the server, where the emotion engine analyzes it to recognize the user's emotions.

[1436] Input: Voice or text of the user's question or request

[1437] Output: Perceived emotional state (e.g., positive, negative, neutral)

[1438] Specific operation: The server uses an emotion analysis algorithm to analyze the user's text and voice data and determine their emotional state.

[1439] Step 6: Recommending the best content

[1440] The server uses the results of the trained machine learning model and emotion engine to recommend the most suitable content to the user, and the recommended content is tailored based on the user's current emotional state.

[1441] Input: Perceived emotional state and user profile

[1442] Output: Recommended content

[1443] Specific operation: The server uses a machine learning model to select the optimal content based on user data and emotional data, and sends the results to the device.

[1444] Step 7: Provide to users

[1445] The terminal displays the most suitable content provided by the server to the user, who can then view the recommended content and request more detailed information.

[1446] Input: Recommended content provided by the server

[1447] Output: Content information displayed on the device

[1448] Specific operation: The device uses a user interface to display the recommended content and allow the user to access it.

[1449] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the control target 443 to output the result of the specific processing. The microphone 238 acquires voice indicating a user input regarding the result of the specific processing. The control unit 46A transmits voice data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the voice data.

[1450] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1451] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the robot 414.

[1452] The emotion identification model 59 as an emotion engine may determine the user's emotion according to a specific mapping. Specifically, the emotion identification model 59 may determine the user's emotion according to an emotion map (see FIG. 9), which is a specific mapping. Similarly, the emotion identification model 59 may determine the robot's emotion, and the identification processing unit 290 may perform identification processing using the robot's emotion.

[1453] FIG. 9 is a diagram illustrating an emotion map 400 on which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. Emotions closer to the center of the concentric circles are more primitive. Emotions representing states and actions arising from a state of mind are arranged on the outer edges of the concentric circles. The concept of emotion includes both affect and mental states. Emotions generally generated from reactions occurring in the brain are arranged on the left side of the concentric circles. Emotions generally induced by situational judgment are arranged on the right side of the concentric circles. Emotions generally generated from reactions occurring in the brain and induced by situational judgment are arranged on the upper and lower sides of the concentric circles. Furthermore, the emotion of "pleasure" is arranged on the upper side of the concentric circles, and the emotion of "discomfort" is arranged on the lower side. In this way, in the emotion map 400, multiple emotions are mapped based on the structure by which emotions are generated, and emotions that tend to occur simultaneously are mapped close to each other.

[1454] These emotions are distributed in the 3 o'clock direction on emotion map 400, and typically fluctuate between relief and anxiety. In the right half of emotion map 400, situational awareness dominates over internal sensations, resulting in a sense of calm.

[1455] The inside of emotion map 400 represents what is going on in the mind, and the outside of emotion map 400 represents behavior, so the further you go outside emotion map 400, the more visible the emotions become (the more they are expressed in behavior).

[1456] Human emotions are based on various balances, such as posture and blood sugar levels. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. Emotions can also be created for robots, automobiles, and motorcycles, based on various balances, such as posture and remaining battery life. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. An emotion map can be generated, for example, based on Dr. Mitsuyoshi's emotion map (Research on Voice Emotion Recognition and Emotional Brain Physiological Signal Analysis Systems, Tokushima University, Doctoral Dissertation: https: / / ci.nii.ac.jp / naid / 500000375379). The left half of the emotion map lists emotions belonging to the "reaction" domain, where sensation is dominant. The right half of the emotion map lists emotions belonging to the "situation" domain, where situational awareness is dominant.

[1457] The emotion map defines two emotions that promote learning. One is a negative emotion on the situation side, around the middle of "repentance" or "reflection." In other words, this occurs when the robot experiences negative emotions such as "I never want to feel this way again" or "I don't want to be scolded again." The other is a positive emotion on the response side, around "desire." In other words, this occurs when the robot experiences positive feelings such as "I want more" or "I want to know more."

[1458] The emotion identification model 59 inputs user input into a pre-trained neural network, obtains emotion values ​​indicating each emotion shown in the emotion map 400, and determines the user's emotion. This neural network is pre-trained based on multiple pieces of training data that are combinations of user input and emotion values ​​indicating each emotion shown in the emotion map 400. Furthermore, this neural network is trained so that emotions that are located close to each other have similar values, as in the emotion map 900 shown in FIG. 10. FIG. 10 shows an example in which multiple emotions, "relieved," "calm," and "reassuring," have similar emotion values.

[1459] The system according to the present disclosure has been described above mainly with respect to the functions of the data processing device 12, but the system according to the present disclosure is not necessarily implemented on a server. The system according to the present disclosure may be implemented as a general information processing system. The present disclosure may be implemented, for example, as a software program running on a personal computer or an application running on a smartphone, etc. The method according to the present disclosure may be provided to users in the form of SaaS (Software as a Service).

[1460] In the above embodiment, an example was given in which the specific processing is performed by one computer 22, but the technology of the present disclosure is not limited to this, and the specific processing may be distributed and performed by a plurality of computers including the computer 22. For example, the data generation model 58 may be provided in an external device of the data processing device 12, and data may be generated in the external device in accordance with input data.

[1461] In the above embodiment, an example in which the specific processing program 56 is stored in the storage 32 has been described, but the technology of the present disclosure is not limited to this. For example, the specific processing program 56 may be stored in a portable, computer-readable, non-transitory storage medium such as a USB (Universal Serial Bus) memory. The specific processing program 56 stored in the non-transitory storage medium is installed in the computer 22 of the data processing device 12. The processor 28 executes the specific processing in accordance with the specific processing program 56.

[1462] Alternatively, the specific processing program 56 may be stored in a storage device such as a server connected to the data processing device 12 via the network 54, and the specific processing program 56 may be downloaded and installed on the computer 22 in response to a request from the data processing device 12.

[1463] It is not necessary to store all of the specific processing program 56 in a storage device such as a server connected to the data processing device 12 via the network 54, or to store all of the specific processing program 56 in the storage 32; only a portion of the specific processing program 56 may be stored.

[1464] The hardware resource for executing a specific process can be any of the following processors: An example of a processor is a CPU, which is a general-purpose processor that functions as a hardware resource for executing a specific process by executing software, i.e., a program. Another example of a processor is a dedicated electrical circuit, such as an FPGA (Field-Programmable Gate Array), a PLD (Programmable Logic Device), or an ASIC (Application Specific Integrated Circuit), which is a processor with a circuit configuration designed specifically for executing a specific process. Each processor has built-in or connected memory, and each processor uses the memory to execute the specific process.

[1465] The hardware resource that executes the specific processing may be configured with one of these various processors, or may be configured with a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Also, the hardware resource that executes the specific processing may be a single processor.

[1466] As an example of a system configured with a single processor, first, one processor is configured by combining one or more CPUs and software, and this processor functions as a hardware resource that executes a specific process. Second, there is a system that uses a processor that realizes the functions of an entire system including multiple hardware resources that execute a specific process on a single IC chip, as typified by SoC (System-on-a-chip). In this way, a specific process is realized using one or more of the above-mentioned various processors as hardware resources.

[1467] Furthermore, the hardware structure of these various processors can be, more specifically, an electric circuit that combines circuit elements such as semiconductor devices. The specific processing described above is merely an example. Therefore, it goes without saying that unnecessary steps may be deleted, new steps may be added, or the processing order may be rearranged, without departing from the spirit of the invention.

[1468] The above-described description and illustrations are a detailed explanation of the parts related to the technology of the present disclosure and are merely an example of the technology of the present disclosure. For example, the above description of the configuration, functions, actions, and effects is an explanation of an example of the configuration, functions, actions, and effects of the parts related to the technology of the present disclosure. Therefore, it goes without saying that unnecessary parts may be deleted, new elements may be added, or replacements may be made to the above-described description and illustrations within the scope of the gist of the technology of the present disclosure. Furthermore, to avoid confusion and facilitate understanding of the parts related to the technology of the present disclosure, the above-described description and illustrations omit explanations of common technical knowledge that do not require particular explanation to enable the implementation of the technology of the present disclosure.

[1469] All publications, patent applications, and technical standards mentioned in this specification are herein incorporated by reference to the same extent as if each individual publication, patent application, or technical standard was specifically and individually indicated to be incorporated by reference.

[1470] The following is further disclosed regarding the above embodiment.

[1471] (Claim 1)

[1472] a means for collecting personal information provided by users;

[1473] A means of integrating and collecting information from multiple account services;

[1474] A means for preprocessing the collected data and inputting it into a learning model;

[1475] a means for training a machine learning model using the preprocessed data; and

[1476] A means for generating an optimal answer to a question from a user;

[1477] means for providing the generated answer to the user;

[1478] A system including:

[1479] (Claim 2)

[1480] The system according to claim 1, wherein the information is obtained through the API of each service.

[1481] (Claim 3)

[1482] 10. The system of claim 1, wherein the system filters unnecessary information and normalizes the data.

[1483] "Example 1"

[1484] (Claim 1)

[1485] a means for collecting personal information provided by users;

[1486] A means of integrating and collecting information from multiple account services;

[1487] A means for preprocessing the collected data and inputting it into a learning model;

[1488] a means for training a machine learning model using the preprocessed data; and

[1489] A means for generating an optimal answer to a question from a user;

[1490] means for providing the generated answer to the user;

[1491] A means for the terminal to acquire authentication information from the user and transmit it to the server;

[1492] A means for the server to obtain user data using the API of each service;

[1493] a means for the server to normalize the data and filter out unwanted data;

[1494] a means for training the preprocessed data using a machine learning algorithm and saving the model;

[1495] means for accepting a user question, generating an answer using the stored model, and transmitting the answer to the terminal;

[1496] A system including:

[1497] (Claim 2)

[1498] 2. The system according to claim 1, wherein the information is obtained through the API of each service, and the terminal transmits the authentication information input by the user to the server.

[1499] (Claim 3)

[1500] The system of claim 1, wherein the server filters out unnecessary information, normalizes the data, and converts the data into a format suitable for the learning model.

[1501] "Application Example 1"

[1502] (Claim 1)

[1503] a means for collecting personal information provided by users;

[1504] A means of integrating and collecting information from multiple account services;

[1505] A means for preprocessing the collected data and inputting it into a learning model;

[1506] a means for training a machine learning model using the preprocessed data; and

[1507] A means for analyzing a user's consumption pattern and generating optimal savings methods and campaign information;

[1508] A means for providing the generated saving methods and campaign information to users;

[1509] A system including:

[1510] (Claim 2)

[1511] The system according to claim 1, wherein the information is obtained through the API of each service.

[1512] (Claim 3)

[1513] 10. The system of claim 1, wherein the system filters unnecessary information and normalizes the data.

[1514] "Example 2: Combining Emotion Engines"

[1515] (Claim 1)

[1516] a means for collecting personal information provided by users;

[1517] A means of integrating and collecting information from multiple account services;

[1518] A means to preprocess the collected data and input it into a machine learning model;

[1519] a means for training a machine learning model using the preprocessed data; and

[1520] A means for generating an optimal answer to a question from a user;

[1521] means for providing the generated answer to the user;

[1522] means for recognizing a user's emotion;

[1523] means for tailoring responses based on user sentiment;

[1524] A system including:

[1525] (Claim 2)

[1526] The system according to claim 1, wherein the information is obtained through the API of each service.

[1527] (Claim 3)

[1528] 10. The system of claim 1, wherein the system filters unnecessary information and normalizes the data.

[1529] "Application example 2 when combining emotion engines"

[1530] (Claim 1)

[1531] a means for collecting personal information provided by users;

[1532] A means of integrating and collecting information from multiple account services;

[1533] A means for preprocessing the collected data and inputting it into a learning model;

[1534] a means for training a machine learning model using the preprocessed data; and

[1535] A means for generating an optimal answer to a question from a user;

[1536] means for providing the generated answer to the user;

[1537] means for analyzing user emotions;

[1538] a means for recommending content based on the results of the sentiment analysis;

[1539] A system including:

[1540] (Claim 2)

[1541] The system according to claim 1, wherein the information is obtained through the API of each service.

[1542] (Claim 3)

[1543] 10. The system of claim 1, wherein the system filters unnecessary information and normalizes the data. [Explanation of symbols]

[1544] 10, 210, 310, 410 Data Processing Systems 12 Data Processing Device 14 Smart Devices 214 Smart Glasses 314 Headset-type terminal 414 Robot< / url:> < / url:> < / url:> < / url:>

Claims

1. a means for collecting personal information provided by users; A means of integrating and collecting information from multiple account services; A means for preprocessing the collected data and inputting it into a learning model; a means for training a machine learning model using the preprocessed data; and A means for generating an optimal answer to a question from a user; means for providing the generated answer to the user; A system including:

2. The system according to claim 1, wherein the information is acquired through an API of each service.

3. 10. The system of claim 1, further comprising filtering of unnecessary information and normalizing the data.

Images