System

Abstract

A system is provided.SOLUTION: A system comprising: means for inputting financial information from a user; means for transmitting the financial information to a server; means for using a generated AI model to analyze the financial information; means for generating customized financial advice with the generated AI model; means for transmitting the customized financial advice to a user device; and means for displaying the customized financial advice at the user device.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The technology of the present disclosure relates to a system. [Background technology]

[0002] Patent document 1 discloses a persona chatbot control method performed by at least one processor, the method including the steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to a description of the chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2022-180282 Summary of the Invention [Problem to be solved by the invention]

[0004] Traditionally, it has been difficult to access individual financial plans, and many people have been unable to receive specific advice on how to save and invest effectively. This has resulted in inadequate financial management, hindering personal financial stability and future asset formation. In addition, some financial advisory services are expensive and unavailable to many users. Furthermore, existing systems have difficulty providing customized advice in real time. There is a need to solve these issues and provide a system that allows anyone to easily receive high-quality financial advice. [Means for solving the problem]

[0005] The present invention provides a system including a means for inputting financial data from a user, a means for transmitting the financial data to a server, a means for using a generative AI model to analyze the financial data, a means for generating customized financial advice using the generative AI model, a means for transmitting the customized financial advice to a user terminal, and a means for displaying the customized financial advice on the user terminal. The system further includes a means for the server to obtain additional financial data using an external API, and a means for the generative AI model to perform analysis taking into account historical data and market trends, thereby enabling more accurate advice to be provided in real time. In this way, users can receive effective and customized savings and investment strategies.

[0006] "User" means an individual or user of the system who inputs financial data and receives analysis results.

[0007] "Financial Data" refers to economic information such as a user's income, expenses, asset status, and risk tolerance.

[0008] "Server" is a computer system that receives financial data and uses generative AI models to analyze the data and generate customized advice.

[0009] A "generative AI model" is a collection of algorithms and programs that use artificial intelligence technology to analyze financial data and generate optimal financial advice for users.

[0010] "Customized financial advice" is specific recommendations, such as savings plans and investment strategies, generated based on a user's individual financial situation and goals.

[0011] "User Device" means a device (e.g., smartphone, tablet, or PC) on which a User can input financial data and receive and display advice sent from the Server.

[0012] An "External API" is an application program interface that a server uses to obtain additional financial data from external financial institutions or services.

[0013] "Market trends" refer to movements or patterns of prices, trading volumes, interest rates, etc. observed in financial markets over a specific period of time.

[0014] "Analysis" is the process of leveraging generative AI models to evaluate data, apply specialized knowledge, and generate optimal advice based on collected financial data. [Brief explanation of the drawings]

[0015] [Figure 1] 1 is a conceptual diagram showing an example of the configuration of a data processing system according to a first embodiment. [Figure 2] 1 is a conceptual diagram showing an example of main functions of a data processing device and a smart device according to a first embodiment. [Figure 3] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a second embodiment. [Figure 4] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and smart glasses according to a second embodiment. [Figure 5] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a third embodiment. [Figure 6] FIG. 11 is a conceptual diagram showing an example of main functions of a data processing device and a headset-type terminal according to a third embodiment. [Figure 7] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a fourth embodiment. [Figure 8] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and a robot according to a fourth embodiment. [Figure 9] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 10] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 11]FIG. 3 is a sequence diagram showing a processing flow of the data processing system according to the first embodiment. [Figure 12] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 1. [Figure 13] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system according to the second embodiment when an emotion engine is combined. [Figure 14] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 2 when an emotion engine is combined. DETAILED DESCRIPTION OF THE INVENTION

[0016] An example of an embodiment of a system according to the technology of the present disclosure will be described below with reference to the accompanying drawings.

[0017] First, the terms used in the following description will be explained.

[0018] In the following embodiments, a coded processor (hereinafter simply referred to as a "processor") may be a single arithmetic device or a combination of multiple arithmetic devices. Furthermore, a processor may be a single type of arithmetic device or a combination of multiple types of arithmetic devices. Examples of arithmetic devices include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), and an APU (Accelerated Processing Unit).

[0019] In the following embodiments, a coded RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a working memory by a processor.

[0020] In the following embodiments, the coded storage is one or more non-volatile storage devices that store various programs, various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), and magnetic tapes.

[0021] In the following embodiments, a communication I / F (Interface) with a symbol is an interface including a communication processor, an antenna, etc. The communication I / F controls communication between multiple computers. Examples of communication standards applied to the communication I / F include wireless communication standards including 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), Bluetooth (registered trademark), etc.

[0022] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." In other words, "A and / or B" means that it may be only A, only B, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" is also applied when three or more things are expressed connected by "and / or."

[0023] [First embodiment]

[0024] FIG. 1 shows an example of the configuration of a data processing system 10 according to the first embodiment.

[0025] 1, a data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0026] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0027] The smart device 14 includes a computer 36, a reception device 38, an output device 40, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The reception device 38, the output device 40, and the camera 42 are also connected to the bus 52.

[0028] The reception device 38 includes a touch panel 38A, a microphone 38B, and the like, and receives user input. The touch panel 38A detects contact with an indicator (for example, a pen or a finger) to receive user input by the touch of the indicator. The microphone 38B detects the user's voice to receive user input by voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0029] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form of expression that the user 20 can perceive (for example, audio and / or text). The display 40A displays visible information such as text and images in accordance with instructions from the processor 46. The speaker 40B outputs audio in accordance with instructions from the processor 46. The camera 42 is a compact digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0030] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54.

[0031] FIG. 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0032] 2, in the data processing device 12, a specific process is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific process is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0033] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0034] In the smart device 14, the processor 46 performs the reception output process. The storage 50 stores a reception output program 60. The reception output program 60 is used in conjunction with the specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0035] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0036] The present invention is a system that allows users to input financial data and uses a generative AI model based on that data to provide customized financial advice. The following describes the processing for each server, terminal, and user in detail.

[0037] Server Processing

[0038] The server first receives financial data (income, expenses, assets, risk tolerance, etc.) submitted by the user. This data is received in real time and stored in an internal database. The server then utilizes external APIs to obtain additional financial data about the user's bank account information and investment portfolio.

[0039] All acquired data is consolidated and cleansed of missing or outlier values. Once the data is clean and standardized, it is fed into a generative AI model. The generative AI model has learned from past market trends and data from other users, and uses this information to generate optimal financial advice for the user. The generated advice is then formatted in text and graphs and sent back to the user's device.

[0040] Terminal handling

[0041] The terminal provides an interface for user input. The user uses a form to enter their income, expenses, and asset information, which is then sent to the server via the terminal. The entered data sends requests as needed to establish communication with the server.

[0042] As financial advice is sent from the server, the device receives it in real time, updates its internal data model, and presents it in a user-friendly format (dashboards and reports). It also provides tools for users to take specific actions, such as setting savings goals or making investments.

[0043] User operations

[0044] Users first input their financial data into the device, including income, expenses, asset status, and risk tolerance, and then send this data to the server. Once the server sends them customized financial advice, the user can review it and decide on specific actions to apply.

[0045] For example, users can set monthly savings amounts based on a proposed savings plan, or purchase specific investment products according to a recommended investment strategy. Users can easily manage these actions on their device and track their progress in real time.

[0046] Specific examples

[0047] For example, User A has an annual income of 6 million yen, monthly living expenses of 200,000 yen, and a moderate risk tolerance. User A enters this data into a terminal and sends it to the server. The server receives the data and uses a generative AI model to generate an optimal savings and investment strategy for User A. Specific advice such as "save 1 million yen per year and invest 500,000 yen in a risk-diversified fund" is generated.

[0048] The server then sends this advice to User A's device. User A checks the advice on the device, sets savings goals through the app, and makes investments. This allows User A to efficiently manage their finances.

[0049] In this manner, the present invention is a system that enables users to receive and implement effective, customized financial advice.

[0050] The processing flow will be explained below.

[0051] Server processing flow

[0052] Step 1:

[0053] The server receives financial data sent from the user terminal, including income, expenses, asset status, risk tolerance, etc.

[0054] Step 2:

[0055] The server uses external APIs to obtain additional financial data about the user, such as bank account details and investment portfolios, and receives the response of the external API and stores it in an internal database.

[0056] Step 3:

[0057] The server integrates the received financial data with the additional financial data it has acquired. After the data integration, it performs a cleansing process to correct missing and outlier values.

[0058] Step 4:

[0059] The server standardizes the cleansed data and inputs it into the generative AI model. The standardization process includes scaling numerical data and encoding categorical data.

[0060] Step 5:

[0061] A generative AI model analyzes input data and generates optimal savings and investment strategies for users, taking into account historical market trends and data from other users.

[0062] Step 6:

[0063] The server converts the generated advice into text and graphical formats, making it easily understandable to the user.

[0064] Step 7:

[0065] The server then sends the prepared advice to the user's device, securely using the HTTPS protocol.

[0066] Terminal processing flow

[0067] Step 1:

[0068] The terminal displays a form for the user to enter financial data, including income, expenses, asset information, and risk tolerance.

[0069] Step 2:

[0070] As users enter data into forms, the terminal validates the input data in real time and prompts the user to re-enter any incomplete data.

[0071] Step 3:

[0072] The terminal sends the validated data to the server using the HTTPS protocol to ensure the safety of the user data.

[0073] Step 4:

[0074] When financial advice is sent from the server, the terminal receives it and converts the received data into an internal data model.

[0075] Step 5:

[0076] The device visualizes the advice it receives and displays it in the form of a dashboard or report that is easy for the user to understand.

[0077] Step 6:

[0078] The terminal provides an interface that allows users to take specific actions based on the advice (such as setting savings goals or making investments).

[0079] User Process Flow

[0080] Step 1:

[0081] The user accesses the terminal and inputs their income, expenses, asset status, and risk tolerance, entering the required information into the input form one by one.

[0082] Step 2:

[0083] Once the input is complete, the user presses the submit button to send the data to the server, and a notification message is displayed when the data is sent.

[0084] Step 3:

[0085] When the advice is sent from the server, the user checks it on the device, understands the advice, and prepares by taking notes of any necessary parts.

[0086] Step 4:

[0087] Users take specific financial actions based on the advice, such as setting up regular savings or purchasing investment products.

[0088] Step 5:

[0089] Users frequently check the progress of their actions on their devices and make replanning and adjustments as needed.

[0090] Step 6:

[0091] Users can provide feedback on their achieved goals and ongoing plans via the device, and ask for further advice. This feedback will help provide more accurate advice.

[0092] Example 1

[0093] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0094] Conventional financial advice systems face the problem of being unable to provide adequately customized advice based on a user's individual financial data. In particular, complex processes are required, such as real-time data integration and cleansing, and the acquisition of additional data using external APIs. However, because these processes are not sufficiently optimized, users are unable to receive highly accurate advice.

[0095] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0096] In this invention, the server includes means for inputting financial data from a user, means for transmitting the financial data to the server, means for the server to obtain additional financial data using an external API, means for integrating the financial data with the additional financial data and cleansing missing values ​​and outliers, means for generating customized financial advice using the generative AI model, means for transmitting the customized financial advice to a user terminal, and means for displaying the customized financial advice on the user terminal, thereby enabling more accurate customized financial advice to be provided to users in real time.

[0097] A "user" is a person or entity that inputs financial data and receives customized financial advice.

[0098] "Financial data" is a general term for information about a user's financial situation, such as income, expenses, asset status, and risk tolerance.

[0099] A "server" is a computer system for receiving and processing financial data submitted by users.

[0100] "External API" means an application program interface for accessing third-party services to obtain additional financial data.

[0101] "Financial data" is a general term for information about bank accounts and investment portfolios.

[0102] "Integration" refers to the process of bringing together data obtained from different data sources.

[0103] "Cleansing" is a data preparation process that removes missing and outliers from data and makes any necessary corrections.

[0104] A "generative AI model" is an algorithm or system that uses artificial intelligence to generate customized financial advice.

[0105] "Customized financial advice" is personalized advice created based on a user's individual financial data.

[0106] A "user terminal" is a device through which a user inputs financial data and receives financial advice sent from the server.

[0107] A "prompt" is a string of text data that is input into a generative AI model, and the AI ​​uses this to determine the output it generates.

[0108] The present invention is a system that allows users to input their own financial data and uses a generative AI model to provide customized financial advice. The following describes the processing for each server, terminal, and user in detail.

[0109] Server Processing

[0110] The server first receives financial data (income, expenses, asset status, risk tolerance, etc.) sent by the user. This data is received in real time and stored in an internal database, for example, using a relational database such as MySQL or PostgreSQL.

[0111] The server then uses external APIs to retrieve additional financial data, such as the user's bank account information and investment portfolio, using common APIs that are widely used for financial information.

[0112] All acquired data is consolidated and processed to remove missing or outlier values. Data cleansing is performed using Python or R libraries (such as Pandas). Once the data is clean and standardized, it is input into a generative AI model. This generative AI model is a natural language generation model, such as GPT-3 or BERT, which has been trained on past market trends and data from other users. Based on this, it generates optimal financial advice for the user. The generated advice is then formatted in text or graph form, visualized using tools such as Matplotlib, and sent back to the user's device.

[0113] Terminal handling

[0114] The terminal provides an interface for user input. The user uses a form to enter their income, expenses, and asset information, which is then sent to the server via the terminal. The entered data is sent to the server in real time using AJAX requests or WebSockets.

[0115] As financial advice is sent from the server, the device receives it in real time and updates its internal data model. It builds a dashboard using front-end frameworks such as React or Angular to display the advice in a format that's easy for users to understand. It also provides tools for users to take specific actions, such as setting savings goals or making investments. This includes interfaces for taking actions directly using APIs such as Twilio and Stripe.

[0116] User operations

[0117] Users first input their financial data into the device, including income, expenses, asset status, and risk tolerance, and then send this data to the server. Once the server sends them customized financial advice, the user can review it and decide on specific actions to apply.

[0118] For example, users can set monthly savings amounts based on a proposed savings plan, or purchase specific investment products according to a recommended investment strategy. Users can easily manage these actions on their device and track their progress in real time. This includes tracking user behavior using analytics tools such as Google Analytics and Mixpanel.

[0119] Specific examples

[0120] For example, User A has an annual income of 6 million yen, monthly living expenses of 200,000 yen, and a moderate risk tolerance. User A enters this data into a device and sends it to the server. The server receives the data and uses a generative AI model to generate an optimal savings and investment strategy for User A. The advice will be specific, such as "save 1 million yen per year and invest 500,000 yen in a risk-diversified fund."

[0121] An example of a prompt sentence to input to the generative AI model is as follows:

[0122] "User A's annual income is 6 million yen, and his monthly living expenses are 200,000 yen. User A has a medium risk tolerance. Based on this, please suggest the optimal savings and investment strategy."

[0123] This prompt is then fed into a generative AI model to generate specific financial advice for the user, helping them to manage their finances more efficiently.

[0124] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0125] Step 1: User enters financial data

[0126] The user enters their income, expenses, asset status, and risk tolerance into a form on the device, using text fields, drop-down lists, check boxes, etc.

[0127] input:

[0128] Income: 6 million yen

[0129] Expenses: 200,000 yen per month

[0130] Asset status: 5 million yen

[0131] Risk tolerance: Medium

[0132] output:

[0133] This financial data is entered into the terminal and converted into JSON format.

[0134] Specific behavior:

[0135] When the user clicks the "Submit" button on the form, the data is packaged in JSON format.

[0136] Step 2: Sending and Receiving Data

[0137] The terminal sends the entered financial data to the server, which receives it and stores it in a database. Real-time transmission is achieved using AJAX requests and WebSockets.

[0138] input:

[0139] Financial data in JSON format: {Annual income: $6,000,000, Monthly living expenses: $2,000, Asset status: $5,000, Risk tolerance: Medium}

[0140] output:

[0141] The server stores the data in a database.

[0142] Specific behavior:

[0143] The terminal sends JSON data, which the server receives and stores in a MySQL database.

[0144] Step 3: Obtaining additional data via an external API

[0145] The server uses an external API to retrieve the user's bank account information and investment portfolio data. It uses a financial information API (e.g., Plaid).

[0146] input:

[0147] User credentials

[0148] output:

[0149] Additional financial data (bank account balances, investment portfolios, etc.)

[0150] Specific behavior:

[0151] The server sends a request to the external API and stores the retrieved data back in the database.

[0152] Step 4: Integrate and cleanse the data

[0153] The server integrates all data it has acquired and cleanses missing and outlier values. It uses the Python Pandas library.

[0154] input:

[0155] Financial Data and Additional Financial Data

[0156] output:

[0157] Clean and standardized datasets

[0158] Specific behavior:

[0159] Run the Python script to load the data into a data frame and impute missing values.

[0160] Step 5: Generative AI model generates financial advice

[0161] The server inputs the cleansed data into a generative AI model, such as GPT-3, to generate optimal financial advice.

[0162] input:

[0163] Clean and standardized datasets

[0164] output:

[0165] Generated financial advice (e.g., "Save 1 million yen per year and invest 500,000 yen in a risk-diversified fund")

[0166] Specific behavior:

[0167] Prompt sentences are generated and input into a generative AI model to generate advice.

[0168] Step 6: Submitting and viewing advice

[0169] The server sends the generated financial advice to the user's device, which receives it and displays it on a dashboard. This uses React and Angular, among other tools.

[0170] input:

[0171] Generated Financial Advice

[0172] output:

[0173] Personalized financial advice displayed in a dashboard

[0174] Specific behavior:

[0175] The server sends advice in JSON format, which is displayed on the device's dashboard.

[0176] Step 7: Performing User Actions

[0177] Users can set and take specific actions based on the financial advice they receive, such as setting savings goals or making investments.

[0178] input:

[0179] Actions taken by the user

[0180] output:

[0181] The action taken and its result

[0182] Specific behavior:

[0183] Actions are executed when the user clicks on the dashboard's "Set Savings Goals" or "Invest" buttons. In some cases, Twilio or Stripe APIs are used.

[0184] (Application example 1)

[0185] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0186] In conventional financial management systems, it was difficult for users to input financial data and receive appropriate financial advice based on that data. Furthermore, the lack of means for users to set individual savings goals or run investment simulations made it difficult to provide optimal financial management for each individual user. The present invention aims to solve these problems and provide a system that allows users to easily and effectively manage their finances.

[0187] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0188] In this invention, the server includes means for inputting financial data from a user, means for transmitting the financial data to the server, means for using a generative AI model to analyze the financial data, means for generating customized financial advice using the generative AI model, means for transmitting the customized financial advice to a user terminal, means for displaying the customized financial advice on the user terminal, and means for setting savings goals and executing investment simulations on the user terminal, thereby enabling users to intuitively operate the system, receive customized financial advice, and effectively manage their savings and investments.

[0189] "User" means an individual or legal entity that uses the system, inputs financial data, and receives advice.

[0190] "Financial Data" refers to financial information such as a user's income, expenses, asset status, and risk tolerance.

[0191] "Server" means a computer system that receives financial data submitted by users, utilizes external APIs to obtain additional financial data, analyzes the data, and generates advice.

[0192] A "generative AI model" is a machine learning algorithm that analyzes a user's financial data and generates customized financial advice.

[0193] "Customized financial advice" refers to personalized investment strategies and savings plans created by generative AI models based on a user's financial data.

[0194] "User Device" means the device (e.g., smartphone, tablet, PC, etc.) used by a User to input financial data and view and interact with customized financial advice.

[0195] "External API" means an application program interface used by the Server to obtain additional financial data from external financial institutions or market data providers.

[0196] A "savings goal" refers to a specific savings amount and savings period set by a user, and is a numerical target that the user should aim for financially.

[0197] "Investment simulation" is a process of estimating future returns and risks in advance based on the amount of investment expected by the user.

[0198] "Data analysis" refers to the process of cleaning up financial and external data collected by the server, standardizing it, and inputting it into a generative AI model.

[0199] This invention is a system that allows users to input financial data and uses a generative AI model based on that data to provide customized financial advice. Below, we will explain in detail the processing for each server, terminal, and user.

[0200] Server Processing

[0201] The server receives financial data (income, expenses, asset status, risk tolerance, etc.) sent by the user. This data is received in real time and stored in an internal database. The server then uses external APIs to obtain additional financial data about the user's bank account information and investment portfolio. All of the obtained data is consolidated and cleansed of missing values ​​and outliers. Once the data is clean and standardized, it is input into a generative AI model. The generative AI model has learned from past market trends and data from other users and generates optimal financial advice for the user based on this. The generated advice is formatted in text and graphs and sent back to the user's device.

[0202] Terminal handling

[0203] The terminal provides an interface for user input. The user uses a form to enter their income, expenses, and asset information, which is then sent via the terminal to the server. The entered data sends requests and establishes communication with the server as needed. As financial advice is sent from the server, the terminal receives it in real time and updates its internal data model. It not only displays the advice in a user-friendly format (dashboards and reports), but also provides tools for the user to take specific actions (e.g., setting savings goals or simulating investments).

[0204] User operations

[0205] Users first enter their financial data into the device, including income, expenses, asset status, and risk tolerance, and then send this data to the server. After receiving customized financial advice from the server, users review it and decide on specific actions to apply. For example, they can set a monthly savings amount based on the proposed savings plan, or purchase specific investment products according to the recommended investment strategy. Users can easily manage these actions on the device and track their progress in real time.

[0206] Specific examples

[0207] For example, User A has an annual income of 6 million yen, monthly living expenses of 200,000 yen, and a moderate risk tolerance. User A enters this data into his device and sends it to the server. The server receives the data and uses a generative AI model to generate an optimal savings and investment strategy for User A. Specific advice is generated, such as "save 1 million yen per year and invest 500,000 yen in a risk-diversified fund." The server then sends this advice to User A's device. User A checks the advice on his device, sets actual savings goals through the app, and makes investments. This allows User A to efficiently manage his finances.

[0208] Hardware and software used

[0209] Hardware: Smartphones, tablets, computers

[0210] Software: Python, external APIs (e.g., APIs for retrieving financial data), generative AI models (e.g., GPT-4)

[0211] Prompt Sentence Examples

[0212] "User income: 6 million yen, monthly living expenses: 200,000 yen, assets: 1 million yen, risk tolerance: medium. Based on these criteria, please suggest the optimal savings plan and investment strategy for the user."

[0213] In this manner, the present invention is a system that enables users to receive and implement effective, customized financial advice.

[0214] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0215] Step 1:

[0216] The user enters their financial data into the terminal, including income, expenses, assets, risk tolerance, etc. The data from the user input form is then ready to be sent to the server.

[0217] Input: User's income, expenses, financial situation, risk tolerance

[0218] Output: Financial data entered into the terminal

[0219] Step 2:

[0220] The terminal sends the user's input data to the server, which receives the data in real time and stores it in an internal database.

[0221] Input: Financial data entered into the terminal

[0222] Output: Financial data stored on the server

[0223] Step 3:

[0224] The server uses external APIs to retrieve additional financial data about the user, such as bank account details and investment portfolio, which is then integrated with the existing user data.

[0225] Input: Financial data stored on the server, plus additional financial data retrieved from external APIs

[0226] Output: Consolidated comprehensive user financial data

[0227] Step 4:

[0228] The server cleanses the consolidated financial data and corrects missing and outlier values, ensuring clean and standardized input data for generative AI models.

[0229] Input: Integrated comprehensive user financial data

[0230] Output: Clean, cleaned financial data

[0231] Step 5:

[0232] The server then inputs the cleansed data into a generative AI model, which has learned about past market trends and data from other users, and generates optimal financial advice for the user based on this data.

[0233] Input: Clean, cleaned financial data

[0234] Output: Customized financial advice from a generative AI model

[0235] Step 6:

[0236] The server then formats the generated financial advice in text and graph format and sends it back to the user's terminal, which receives it in real time and displays it in a format that is easy for the user to understand.

[0237] Input: Customized financial advice from a generative AI model

[0238] Output: Financial advice displayed on the terminal

[0239] Step 7:

[0240] The terminal provides tools for users to set savings goals and run investment simulations. Users can set specific savings goals and investment simulations and execute plans based on them.

[0241] Input: Financial advice displayed on user terminal

[0242] Output: Savings goals and investment simulation plans set by the user

[0243] Step 8:

[0244] Users can access financial advice and take concrete actions on savings and investments through their devices, and the app tracks progress in real time, enabling them to effectively manage their finances.

[0245] Input: Savings goals and investment simulation plans set by the user

[0246] Output: Administrative data on savings and investment actions taken

[0247] Through these steps, the present invention enables users to receive and implement effective, customized financial advice.

[0248] Furthermore, an emotion engine that estimates the user's emotion may be combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59 and perform identification processing using the user's emotion.

[0249] This invention is a system that allows users to input financial data and emotional data, and then provides customized financial advice based on that data using a generative AI model and an emotional engine. The following describes the processing for each server, terminal, and user in detail.

[0250] Server Processing

[0251] 1. Data reception:

[0252] The server first receives financial data (income, expenditure, asset status, risk tolerance, etc.) and emotional data (text messages, voice data, facial expression data, etc.) sent from the user's device. This data is sent to the server in real time and stored in an internal database.

[0253] 2. Data Integration:

[0254] The server uses external APIs to retrieve additional financial data about the user's bank account information and investment portfolio, and then integrates it with the received financial data. After data integration, it performs a cleansing process to correct missing values ​​and outliers.

[0255] 3. Emotion analysis:

[0256] The server uses an emotion engine to analyze the received emotion data, which identifies the user's emotional state (e.g., stress, relief, interest, etc.) and stores it in a database.

[0257] 4. Data Normalization and Analysis:

[0258] The cleansed data is standardized and fed into a generative AI model that takes into account market trends and past user data to generate optimal savings plans and investment strategies for users based on financial and sentiment analysis data.

[0259] 5. Advice generation and transmission:

[0260] The generated advice is formatted as text and graphs and sent to the user's device in an appropriate communication format (e.g., encouraging words or detailed explanations) based on the results of sentiment analysis.

[0261] Terminal handling

[0262] 1. Data Entry:

[0263] The terminal provides a form for users to input financial data and emotional data, including income, expenses, asset information, and risk tolerance, and emotional data including text messages, voice data, and facial expression data.

[0264] 2. Real-time validation and submission:

[0265] Once financial and emotional data is entered, the terminal validates it in real time, prompting the user to re-enter any incomplete data, and then sends the validated data to the server.

[0266] 3. Advice Receiving and Display:

[0267] The terminal receives customized financial advice sent from the server and converts it into an internal data model. The terminal visualizes the received advice and displays it in the form of a dashboard or report. Based on the results of sentiment analysis, the terminal provides advice to the user in an appropriate tone and format.

[0268] 4. Take action:

[0269] The device provides an interface for users to take specific actions based on the advice (such as setting savings goals or making investments), and the progress of the actions taken by the user is updated in real time on the device.

[0270] User operations

[0271] 1. Data Entry:

[0272] Users access the terminal and input their financial data (income, expenses, asset status, risk tolerance) and emotional data (text, voice, facial expressions).

[0273] 2. Data transmission:

[0274] Once the input is complete, the user presses the submit button to send the data to the server, and a notification message is displayed when the data is sent.

[0275] 3. Advice confirmation:

[0276] The user checks the customized financial advice sent from the server along with additional information based on sentiment analysis. For example, if the advice is to "save 1 million yen a year and invest 500,000 yen in a risk-diversified fund," an encouraging message and explanation of the risks will be added based on the results of sentiment analysis.

[0277] 4. Action execution:

[0278] Users can then take specific financial actions based on the advice, such as setting up regular savings or purchasing investment products, via their device.

[0279] 5. Feedback and Replanning:

[0280] The user frequently checks the progress of the action on the device, replans if necessary, and receives feedback based on their emotional state to help generate advice for the next action.

[0281] Specific examples

[0282] For example, User B has an annual income of 8 million yen, monthly living expenses of 250,000 yen, and a high risk tolerance. User B enters this financial data and emotional data, such as "I've been feeling stressed lately," into his / her device and sends it to the server. The server receives this data and analyzes it using a generative AI model and an emotion engine. As a result, it generates advice that is optimal for User B, such as "invest 1.5 million yen per year in a risk-diversified fund and put 500,000 yen into short-term savings." Furthermore, based on the results of the emotion analysis, it also provides "suggestions for low-risk investment strategies to reduce stress" and "detailed explanations to give a sense of security about investing."

[0283] In this way, the present invention provides customized financial advice that takes into account a user's overall financial situation and emotional state, helping users manage their finances more effectively.

[0284] The processing flow will be explained below.

[0285] Server processing flow

[0286] Step 1:

[0287] The server receives financial data and emotional data in real time from the user terminal. The financial data includes income, expenditure, asset status, and risk tolerance, and the emotional data includes text messages, voice data, and facial expression data.

[0288] Step 2:

[0289] The server uses external APIs to obtain additional financial data about the user's bank account details and investment portfolio, which is then stored in an internal database.

[0290] Step 3:

[0291] The server integrates the received financial data with the additional financial data it acquires, and performs a cleansing process to correct missing or outlier values.

[0292] Step 4:

[0293] The server uses an emotion engine to analyze the received emotion data, extracting emotion tags (e.g., stress, relief, interest, etc.) from text messages and voice data.

[0294] Step 5:

[0295] The server standardizes the cleansed financial data and feeds it, along with sentiment analysis results, into the generative AI model. The standardization process includes scaling numerical data and encoding categorical data.

[0296] Step 6:

[0297] The generative AI model analyzes the input data and generates optimal savings plans and investment strategies for users, and the analysis results are converted into text and graph formats.

[0298] Step 7:

[0299] The server then adds a message with an appropriate tone based on the results of sentiment analysis to the generated advice. For example, if a user is under stress, the server might recommend "low-risk investments" and add words of reassurance.

[0300] Step 8:

[0301] The server then sends the prepared advice to the user's device, securely using the HTTPS protocol.

[0302] Terminal processing flow

[0303] Step 1:

[0304] The terminal displays a form for users to enter financial and emotional data, including income, expenses, asset information, risk tolerance, and emotional data (text messages, voice, and facial expressions).

[0305] Step 2:

[0306] As users enter data into forms, the terminal validates the input data in real time and prompts the user to re-enter the data if it is incomplete.

[0307] Step 3:

[0308] The validated data is sent to the server securely using the HTTPS protocol.

[0309] Step 4:

[0310] The terminal receives customized financial advice sent from the server and converts the received data into an internal data model.

[0311] Step 5:

[0312] The device visualizes and displays advice to the user in the form of a dashboard or report, and delivers messages in an appropriate tone based on the results of sentiment analysis.

[0313] Step 6:

[0314] Providing an interface for users to take specific actions based on advice, including interfaces to help set savings goals and make investments.

[0315] Step 7:

[0316] The progress of actions performed by the user is updated in real time and displayed on the device.

[0317] User Process Flow

[0318] Step 1:

[0319] Users access the terminal and input their financial data (income, expenses, asset status, risk tolerance) and emotional data (text messages, voice, facial expressions).

[0320] Step 2:

[0321] Once the input is complete, the user presses the submit button to send the data to the server, and a notification message is displayed when the data has been sent.

[0322] Step 3:

[0323] The user checks the customized financial advice sent from the server and messages based on sentiment analysis. For example, the advice might be "save 1 million yen a year and invest 500,000 yen in a risk-diversified fund," along with suggestions for low-risk investment strategies to reduce stress and encouraging messages.

[0324] Step 4:

[0325] Users take specific financial actions based on the advice, such as setting up regular savings or purchasing investment products.

[0326] Step 5:

[0327] Users frequently check the progress of their actions on their devices and make replanning and adjustments as needed.

[0328] Step 6:

[0329] Users can provide feedback on their achieved goals and ongoing plans via their devices, which will be used to generate the next advice. This feedback will be used to provide even more accurate advice.

[0330] Example 2

[0331] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0332] In modern financial management, not only individual users' financial data but also their emotional state at any given time is an important factor. However, conventional financial advisory systems have been unable to fully utilize emotional data, making it difficult to provide customized advice. This has led to issues such as users feeling stressed when making appropriate financial management and investment decisions, making it difficult to manage their finances efficiently.

[0333] The identification process by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means. In this invention, the server includes: means for inputting financial data and emotion data from a user; means for transmitting the financial data and emotion data to the server; means for using a generative AI model and an emotion engine to analyze the data; means for generating customized financial advice using the generative AI model and emotion engine; means for transmitting the customized financial advice to a user terminal; means for displaying the customized financial advice on the user terminal; means for the server to obtain additional financial data using an external API and integrate the data; means for the server to analyze the emotion data and identify the user's emotional state; means for the user terminal to validate the financial data and emotion data in real time and notify the user of any input errors; means for the customized financial advice to be provided to the user in a form that reflects the emotion analysis results; means for the user terminal to support the user in taking action based on the advice; and means for the user to provide feedback that is used to generate the next advice. This enables more personalized financial management for users and provides optimal advice that takes into account their emotional state and market trends.

[0334] markdown

[0335] A "user" is an individual or entity that uses the system to input financial and emotional data and receive customized financial advice.

[0336] "Financial data" refers to data relating to the user's economic situation, such as income, expenses, asset status, and risk tolerance.

[0337] "Emotion data" refers to data such as text messages, voice data, and facial expression data that indicate the user's emotional state.

[0338] A "server" is a central computing unit that receives data sent from user terminals, performs analysis, and generates financial advice.

[0339] A "generative AI model" is a machine learning model that takes into account market trends and historical data to generate customized financial advice based on input data.

[0340] An "emotion engine" is a software module for analyzing emotion data and identifying the user's emotional state.

[0341] A "user terminal" is an electronic device through which a user inputs financial and emotional data and receives customized financial advice transmitted from a server.

[0342] An "external API" is an interface for obtaining data from other systems or services.

[0343] "Real-time validation" is a process that immediately checks input data and prompts correction of any incomplete data.

[0344] "Customized financial advice" is advice generated by a generative AI model and emotion engine based on a user's individual financial situation and emotional state.

[0345] A "means for supporting behavioral execution" is a part of the system that provides an interface or functionality for users to take specific actions based on customized financial advice.

[0346] "Feedback" refers to opinions and evaluations provided by users to the system, and is information that will be used to generate the next piece of advice.

[0347] MODE FOR CARRYING OUT THE INVENTION

[0348] The present invention is a system for providing optimal financial advice to users based on financial data and emotional data. The processing performed by the server, terminal, and user will be described in detail below.

[0349] Server Processing

[0350] Data reception

[0351] The server first receives financial data (income, expenses, asset status, risk tolerance, etc.) and emotional data (text messages, voice data, facial expression data, etc.) sent from the user's device. This data is sent to the server in real time and stored in an internal database. For example, if a user sends data from their device such as "monthly income of 500,000 yen, monthly expenses of 300,000 yen," this information is stored in the receiving buffer.

[0352] Data Integration

[0353] The server uses external APIs to retrieve additional financial data about the user's bank account information and investment portfolio, and then integrates it with the original financial data. This process also includes a cleansing process to automatically correct missing data and outliers. For example, if a user has multiple accounts and their account information is retrieved via an API, the collected data can be integrated with the input data.

[0354] Emotion analysis

[0355] The server uses an emotion engine to analyze the received emotion data. Through this analysis, the user's emotional state (e.g., stress, relief, interest, etc.) is identified and stored in a database. For example, from a user's message such as "I'm tired today," the emotion engine determines the user's stress level.

[0356] Data normalization and analysis

[0357] The cleansed data is then standardized and fed into a generative AI model, which takes into account market trends and historical data to generate optimal savings plans and investment strategies for users based on financial and sentiment data. For example, historical market data could be used to suggest optimal investment options for users.

[0358] Advice generation and delivery

[0359] The generated advice is then formatted as text and graphs and sent to the user in an appropriate communication format based on the results of the emotion analysis. For example, advice containing encouraging words and detailed explanations may be sent.

[0360] Terminal handling

[0361] Data Entry

[0362] The device provides a form for users to input financial and emotional data. Financial data includes income, expenses, asset information, and risk tolerance, while emotional data includes text messages, voice data, and facial expression data. For example, the device has text boxes for inputting "income" and "expenses," as well as voice recording and facial expression recognition functions for expressing "emotions."

[0363] Real-time validation and submission

[0364] When financial and emotional data is entered, the terminal validates it in real time and displays a warning if the data is incomplete. For example, if a negative income value is entered, an error message is displayed prompting the user to correct it. The terminal also sends the validated data to the server.

[0365] Advice reception and display

[0366] The device receives customized financial advice sent from the server and displays it visually, including graphical dashboards and detailed text messages. For example, if the advice is to "save 1 million yen a year and invest 500,000 yen in a risk-diversified fund," it will visualize it in pie charts and bar graphs.

[0367] Action Execution

[0368] The device provides an interface for users to take action based on the advice. For example, a "Set Savings Goal" button or an "Invest" button are displayed, and users can tap the button to start taking specific action.

[0369] User operations

[0370] Data Entry

[0371] The user uses the device to input their own financial data (income, expenditure, asset status, risk tolerance) and emotional data (text, voice, facial expression). For example, they input "income 500,000 yen, expenditure 300,000 yen" and also emotional data such as "I've been under a lot of stress lately."

[0372] Data transmission

[0373] The data you have entered will be sent to the server. When you press the send button, the progress will be displayed and you will be notified when the data has been sent.

[0374] Advice confirmation

[0375] Check the customized advice sent from the server. For example, "Invest 1.5 million yen a year in a risk-diversified fund and put 500,000 yen into short-term savings," along with an encouraging message based on the results of sentiment analysis.

[0376] Action execution

[0377] The user then takes specific actions based on the advice, such as purchasing an investment fund, and checks the progress of the action on the device.

[0378] Feedback and replanning

[0379] Users can track the progress of their actions, reorganize their plans if necessary, and provide feedback on their emotional state to help inform the next recommendation. For example, they can provide feedback like, "I'm happy with my current investments, but I'd like to see a less risky option," which will be reflected in the next recommendation.

[0380] Specific examples

[0381] For example, User B has an annual income of 8 million yen, monthly living expenses of 250,000 yen, and a high risk tolerance. User B enters this financial data and emotional data, such as "I've been feeling stressed lately," into his / her device and sends it to the server. The server analyzes the received data and generates advice such as "invest 1.5 million yen per year in a risk-diversified fund and put the remaining 500,000 yen into short-term savings." Furthermore, based on the results of the emotional analysis, the advice is accompanied by "low-risk investment strategies to reduce stress" and "detailed explanations to give a sense of security about investing." This advice is displayed on the device, and the user then takes specific action (purchase an investment fund) through the device.

[0382] Prompt Sentence Examples

[0383] "Enter information about your income and expenses, financial situation, and risk tolerance, and provide your recent emotional state via text, voice, and facial expressions."

[0384] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0385] System program processing flow

[0386] Server Processing

[0387] Step 1: Receiving data

[0388] The server receives financial data (income, expenditure, asset status, risk tolerance) and emotional data (text messages, voice data, facial expression data) sent from the user's device. This data is stored in a receiving buffer in real time. For example, information entered by a user as income of 500,000 yen and expenditure of 300,000 yen is saved.

[0389] Input: Financial and emotional data from user devices

[0390] Output: Data stored in the receive buffer

[0391]

[0392] Step 2: Data Integration

[0393] The server uses external APIs to retrieve additional financial data about the user's bank account information and investment portfolio, and combines it with the incoming financial data. This combined data undergoes cleansing processes to correct missing and outlier values. For example, data retrieved from financial institution APIs can be combined with user input data.

[0394] Input: Incoming financial data and financial data obtained from external APIs

[0395] Output: Unified and cleansed data

[0396]

[0397] Step 3: Sentiment Analysis

[0398] The server uses an emotion engine to analyze the received emotion data. This analysis identifies the user's emotional state (e.g., stress, relief, interest) and stores it in a database. For example, if a user says, "I'm tired today," the emotion engine analyzes the user's stress level.

[0399] Input: Received emotion data

[0400] Output: Parsed emotional state data

[0401]

[0402] Step 4: Data standardization and analysis

[0403] The server standardizes the cleansed data and inputs it into a generative AI model, which takes into account market trends and historical data, and generates optimal savings plans and investment strategies for users based on financial and sentiment data. For example, it might suggest optimal investments based on user data combined with market trend data.

[0404] Input: Cleansed and standardized data

[0405] Output: Analysis results from the generative AI model (savings plan and investment strategy)

[0406]

[0407] Step 5: Advice generation and sending

[0408] The generated advice is then formatted as text and graphs and sent to the user in an appropriate communication format based on the results of the emotion analysis. For example, advice containing encouraging words and detailed explanations may be sent.

[0409] Input: Analysis results and emotional state data

[0410] Output: Customized financial advice sent to the user

[0411] Terminal handling

[0412] Step 1: Data entry

[0413] The device provides a form for users to enter financial and emotional data, including text boxes for entering income and expenses, as well as voice recording and facial recognition capabilities.

[0414] Input: User financial and emotional data input

[0415] Output: Data entered in the input form

[0416]

[0417] Step 2: Real-time validation and submission

[0418] The terminal validates the data entered in real time and displays a warning if the data is incomplete. For example, if a negative income value is entered, an error message is displayed and the user is prompted to correct the data. After validation is complete, the data is sent to the server.

[0419] Input: User-entered financial and sentiment data

[0420] Output: Send validated data to the server

[0421]

[0422] Step 3: Receive and view advice

[0423] The terminal receives customized financial advice sent from the server and displays it in a visualized format (dashboard and graphs). For example, advice such as "save 1 million yen a year and invest 500,000 yen in a risk-diversified fund" is visualized.

[0424] Input: Customized financial advice from the server

[0425] Output: Visualized advice display

[0426]

[0427] Step 4: Take Action

[0428] The device provides an interface for users to take specific actions based on the advice, such as a "Set Savings Goal" button or an "Invest" button that users can tap to start the action.

[0429] Enter: customized financial advice

[0430] Output: User action execution

[0431] User operations

[0432] Step 1: Data entry

[0433] The user inputs financial data (income, expenditure, asset status, risk tolerance) and emotional data (text, voice, facial expression) into the terminal. For example, the user inputs "income 500,000 yen, expenditure 300,000 yen" and emotional data such as "I've been under a lot of stress lately."

[0434] Input: Financial and sentiment data

[0435] Output: Filling in the input form

[0436]

[0437] Step 2: Send data

[0438] The data you have entered will be sent to the server. When you press the send button, the progress will be displayed and you will be notified when the data has been sent.

[0439] Input: Data entered into an input form

[0440] Output: Send data to the server

[0441]

[0442] Step 3: Check the advice

[0443] The user then checks the customized financial advice sent from the server, such as "Invest 1.5 million yen a year in a risk-diversified fund and put 500,000 yen into short-term savings," along with an encouraging message based on the results of sentiment analysis.

[0444] Input: Customized financial advice sent from the server

[0445] Output: Check the visualized advice

[0446]

[0447] Step 4: Take action

[0448] The user then takes specific actions based on the advice, such as purchasing an investment fund, and checks the progress of the action on the device.

[0449] Enter: customized financial advice

[0450] Output: Implementing specific actions

[0451]

[0452] Step 5: Feedback and replanning

[0453] Users can track their progress and adjust their plans as needed. The system also provides feedback based on their emotional state, such as, "I'm happy with my current investments, but I'd like to see a less risky option."

[0454] Input: Action execution results and emotional feedback

[0455] Output: Data used to generate next advice

[0456]

[0457] (Application example 2)

[0458] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0459] In conventional factory work, workers' emotional states have a significant impact on work performance and safety measures, but there is a lack of effective ways to manage them in real time. Furthermore, there is a lack of technology to provide optimal advice to each individual worker. Therefore, there is a need to improve work performance and strengthen safety measures.

[0460] The specific processing by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for inputting financial data and emotional data from a user, means for transmitting the financial data and emotional data to the server, means for using a generative AI model and an emotion engine that analyzes the financial data and emotional data, means for generating customized financial advice and work performance improvement advice using the generative AI model and emotion engine, means for transmitting the customized financial advice and work performance improvement advice to a user terminal, and means for displaying the customized financial advice and work performance improvement advice on the user terminal. This makes it possible to analyze the emotional state of a worker in real time and provide optimal advice.

[0461] "User" refers to a person or organization that uses the system.

[0462] "Financial Data" refers to financial information about an individual or organization, such as income, expenses, assets, and risk tolerance.

[0463] "Emotional data" refers to information that represents an emotional state, such as text messages, voice data, and facial expression data.

[0464] "Server" refers to the central processing unit that receives and analyzes data sent by the User and retransmits the generated information to the User.

[0465] "Generative AI model" refers to a machine learning model that analyzes a user's financial and emotional data to generate customized advice.

[0466] "Emotion engine" refers to an analysis system for analyzing emotion data and identifying a user's emotional state.

[0467] "User terminal" refers to a device used by a user to input data or receive and display information sent from the server.

[0468] "External Data Acquisition Means" refers to means for acquiring and integrating additional information from external data sources.

[0469] "Work data" refers to data related to the progress and performance of work in a factory or on-site.

[0470] "Operation data" refers to data relating to the operating status of machinery and equipment within a factory.

[0471] The present invention is a system that utilizes a user's financial data and emotion data to provide customized financial advice, including work performance improvement and safety measures, using a generative AI model and emotion engine. Specific processing for each server, terminal, and user is described below.

[0472] Server Processing

[0473] 1. Data Reception

[0474] The server receives financial data (e.g., income, expenses, asset status, risk tolerance) and emotional data (e.g., text messages, voice data, facial expression data) sent from the user's device. This data is sent to the server in real time and stored in an internal database.

[0475] 2. External Data Acquisition and Data Integration

[0476] The server uses external data acquisition means to acquire work data and equipment operation data. The acquired data is integrated and cleansed. This process uses data integration programs and data cleaning algorithms in a Python environment.

[0477] 3. Sentiment Analysis and Data Analysis

[0478] The server analyzes the emotion data using an emotion engine (e.g., Spacy, IBM Watson). The analysis results are combined with financial data and input into a generative AI model (e.g., TensorFlow, PyTorch). The generative AI model uses this data to generate customized financial advice and work performance improvement advice.

[0479] 4. Advice Generation and Delivery

[0480] The generated advice is formatted as text and graphs and sent to the user's device in an appropriate communication format (e.g., detailed explanations that provide a sense of security) based on the results of sentiment analysis.

[0481] Terminal handling

[0482] 1. Data Entry

[0483] The device provides an interface for users to input financial and emotional data, including text fields, voice input, and facial expression input using a camera.

[0484] 2. Data Transmission

[0485] The entered data is sent from the terminal to the server. This process includes a data transmission program and validation check function.

[0486] 3. Receiving and displaying advice

[0487] The customized advice sent from the server is received by the device, visualized, and displayed to the user. This user interface includes dashboards and reports. In particular, the advice for improving work performance is displayed on smart glasses.

[0488] User operations

[0489] 1. Data Entry

[0490] The user accesses the input interface of the terminal and inputs their own financial data (e.g., income, expenses, asset status, risk tolerance) and emotional data (e.g., text, voice, facial expression).

[0491] 2. Data Transmission

[0492] Once the input is complete, the user presses the submit button to send the data to the server, and a notification message is displayed when the data is sent.

[0493] 3. Check and implement the advice

[0494] After checking the customized advice sent from the server, the user can take specific actions based on the advice, for example, a worker can adjust their work performance or take a break according to the advice displayed on the smart glasses.

[0495] Specific examples

[0496] For example, if Worker A working in a factory comments that he has been under a lot of stress lately, and this emotional state is also confirmed by facial expression data, the server will generate advice to Worker A based on this, recommending that he prioritize low-risk work and take a temporary break. This advice is displayed on the smart glasses.

[0497] Prompt Sentence Examples

[0498] "Design a real-time emotion analysis application for factory workers. This application requires workers to input their emotion data (e.g., stress or relief) and work performance data, and then displays appropriate advice in real time on smart glasses. Data analysis is based on a generative AI model and an emotion engine. Please also generate the corresponding Python code."

[0499] The present invention is a system that handles emotional data and financial data in an integrated manner to provide customized advice to factory workers, thereby improving productivity and strengthening safety measures.

[0500] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0501] Step 1: The user inputs financial and emotional data into the terminal.

[0502] Users use the device's interface to input financial data such as income, expenses, asset status, and risk tolerance. They also input text messages, voice data, and facial expression data that indicate their emotional state via a camera or microphone. The input data is temporarily stored in the device.

[0503] Step 2: The terminal performs real-time validation and sends the data to the server.

[0504] The terminal validates the entered financial and emotional data in real time to check for incomplete data or outliers. Once validated, the data is encrypted and sent to the server, where it checks the input data (income, expenses, emotional state) and either displays an error message or converts it into a format that can be sent.

[0505] Step 3: The server receives the data and stores it in the database.

[0506] The server receives the financial and emotional data sent from the device and stores it in an internal database, which is used in subsequent processing steps.

[0507] Step 4: The server acquires the additional data using an external data acquisition means.

[0508] The server makes API calls to retrieve work data and equipment operation data from external sources. The retrieved data is integrated with internal data and stored in a database. Additional data (such as equipment operation status) is retrieved and integrated with the existing database.

[0509] Step 5: The server performs the cleansing process.

[0510] The server performs a cleansing process on all the acquired data, filling in missing values ​​and correcting outliers, using a Python data analysis library.

[0511] Step 6: The server performs sentiment analysis.

[0512] The server analyzes the emotional data using an emotion engine (e.g., Spacy, IBM Watson). The analysis results are stored in a database as the user's emotional state (stress, relief, etc.). Input: Emotional data (text, voice, facial expression), Output: Emotional state (stress, elation, etc.).

[0513] Step 7: The server inputs the data into the generative AI model and performs analysis.

[0514] The cleansed data and sentiment analysis results are input into a generative AI model. The generative AI model (e.g., TensorFlow, PyTorch) analyzes the financial data and integrated data to generate customized financial and work performance improvement advice. Input: Financial data, sentiment data, work data. Output: Customized advice (savings plans, investment strategies, work instructions, etc.).

[0515] Step 8: The server sends the generated advice.

[0516] The generated advice is formatted as text and graphs and sent to the user terminal in an appropriate communication format. Input: Advice data (text, graphs), Output: Transmission to the user terminal.

[0517] Step 9: The device receives and displays the advice.

[0518] The user device receives the customized advice sent from the server and visualizes and displays it. The display format is a dashboard or report. In particular, advice for improving work performance is displayed on smart glasses, etc. Input: Received data (advice), Output: Visual display (graph, text).

[0519] Step 10: The user acts on the advice.

[0520] The user checks the customized advice displayed on the device and takes specific actions based on it, such as creating a savings plan or changing the priorities of tasks. Input: Check the advice, Output: Specific actions (savings, investment, task change, etc.).

[0521] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0522] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0523] In the above embodiment, an example in which the specific process is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific process may be performed by the smart device 14.

[0524] [Second embodiment]

[0525] FIG. 3 shows an example of the configuration of a data processing system 210 according to the second embodiment.

[0526] 3, the data processing system 210 includes the data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0527] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0528] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, and the camera 42 are also connected to the bus 52.

[0529] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0530] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0531] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0532] Fig. 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Fig. 4, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0533] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0534] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0535] In the smart glasses 214, the reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0536] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal."

[0537] The present invention is a system that allows users to input financial data and uses a generative AI model based on that data to provide customized financial advice. The following describes the processing for each server, terminal, and user in detail.

[0538] Server Processing

[0539] The server first receives financial data (income, expenses, assets, risk tolerance, etc.) submitted by the user. This data is received in real time and stored in an internal database. The server then utilizes external APIs to obtain additional financial data about the user's bank account information and investment portfolio.

[0540] All acquired data is consolidated and cleansed of missing or outlier values. Once the data is clean and standardized, it is fed into a generative AI model. The generative AI model has learned from past market trends and data from other users, and uses this information to generate optimal financial advice for the user. The generated advice is then formatted in text and graphs and sent back to the user's device.

[0541] Terminal handling

[0542] The terminal provides an interface for user input. The user uses a form to enter their income, expenses, and asset information, which is then sent to the server via the terminal. The entered data sends requests as needed to establish communication with the server.

[0543] As financial advice is sent from the server, the device receives it in real time, updates its internal data model, and presents it in a user-friendly format (dashboards and reports). It also provides tools for users to take specific actions, such as setting savings goals or making investments.

[0544] User operations

[0545] Users first input their financial data into the device, including income, expenses, asset status, and risk tolerance, and then send this data to the server. Once the server sends them customized financial advice, the user can review it and decide on specific actions to apply.

[0546] For example, users can set monthly savings amounts based on a proposed savings plan, or purchase specific investment products according to a recommended investment strategy. Users can easily manage these actions on their device and track their progress in real time.

[0547] Specific examples

[0548] For example, User A has an annual income of 6 million yen, monthly living expenses of 200,000 yen, and a moderate risk tolerance. User A enters this data into a terminal and sends it to the server. The server receives the data and uses a generative AI model to generate an optimal savings and investment strategy for User A. Specific advice such as "save 1 million yen per year and invest 500,000 yen in a risk-diversified fund" is generated.

[0549] The server then sends this advice to User A's device. User A checks the advice on the device, sets savings goals through the app, and makes investments. This allows User A to efficiently manage their finances.

[0550] In this manner, the present invention is a system that enables users to receive and implement effective, customized financial advice.

[0551] The processing flow will be explained below.

[0552] Server processing flow

[0553] Step 1:

[0554] The server receives financial data sent from the user terminal, including income, expenses, asset status, risk tolerance, etc.

[0555] Step 2:

[0556] The server uses external APIs to obtain additional financial data about the user, such as bank account details and investment portfolios, and receives the response of the external API and stores it in an internal database.

[0557] Step 3:

[0558] The server integrates the received financial data with the additional financial data it has acquired. After the data integration, it performs a cleansing process to correct missing and outlier values.

[0559] Step 4:

[0560] The server standardizes the cleansed data and inputs it into the generative AI model. The standardization process includes scaling numerical data and encoding categorical data.

[0561] Step 5:

[0562] A generative AI model analyzes input data and generates optimal savings and investment strategies for users, taking into account historical market trends and data from other users.

[0563] Step 6:

[0564] The server converts the generated advice into text and graphical formats, making it easily understandable to the user.

[0565] Step 7:

[0566] The server then sends the prepared advice to the user's device, securely using the HTTPS protocol.

[0567] Terminal processing flow

[0568] Step 1:

[0569] The terminal displays a form for the user to enter financial data, including income, expenses, asset information, and risk tolerance.

[0570] Step 2:

[0571] As users enter data into forms, the terminal validates the input data in real time and prompts the user to re-enter any incomplete data.

[0572] Step 3:

[0573] The terminal sends the validated data to the server using the HTTPS protocol to ensure the safety of the user data.

[0574] Step 4:

[0575] When financial advice is sent from the server, the terminal receives it and converts the received data into an internal data model.

[0576] Step 5:

[0577] The device visualizes the advice it receives and displays it in the form of a dashboard or report that is easy for the user to understand.

[0578] Step 6:

[0579] The terminal provides an interface that allows users to take specific actions based on the advice (such as setting savings goals or making investments).

[0580] User Process Flow

[0581] Step 1:

[0582] The user accesses the terminal and inputs their income, expenses, asset status, and risk tolerance, entering the required information into the input form one by one.

[0583] Step 2:

[0584] Once the input is complete, the user presses the submit button to send the data to the server, and a notification message is displayed when the data is sent.

[0585] Step 3:

[0586] When the advice is sent from the server, the user checks it on the device, understands the advice, and prepares by taking notes of any necessary parts.

[0587] Step 4:

[0588] Users take specific financial actions based on the advice, such as setting up regular savings or purchasing investment products.

[0589] Step 5:

[0590] Users frequently check the progress of their actions on their devices and make replanning and adjustments as needed.

[0591] Step 6:

[0592] Users can provide feedback on their achieved goals and ongoing plans via the device, and ask for further advice. This feedback will help provide more accurate advice.

[0593] Example 1

[0594] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0595] Conventional financial advice systems face the problem of being unable to provide adequately customized advice based on a user's individual financial data. In particular, complex processes are required, such as real-time data integration and cleansing, and the acquisition of additional data using external APIs. However, because these processes are not sufficiently optimized, users are unable to receive highly accurate advice.

[0596] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0597] In this invention, the server includes means for inputting financial data from a user, means for transmitting the financial data to the server, means for the server to obtain additional financial data using an external API, means for integrating the financial data with the additional financial data and cleansing missing values ​​and outliers, means for generating customized financial advice using the generative AI model, means for transmitting the customized financial advice to a user terminal, and means for displaying the customized financial advice on the user terminal, thereby enabling more accurate customized financial advice to be provided to users in real time.

[0598] A "user" is a person or entity that inputs financial data and receives customized financial advice.

[0599] "Financial data" is a general term for information about a user's financial situation, such as income, expenses, asset status, and risk tolerance.

[0600] A "server" is a computer system for receiving and processing financial data submitted by users.

[0601] "External API" means an application program interface for accessing third-party services to obtain additional financial data.

[0602] "Financial data" is a general term for information about bank accounts and investment portfolios.

[0603] "Integration" refers to the process of bringing together data obtained from different data sources.

[0604] "Cleansing" is a data preparation process that removes missing and outliers from data and makes any necessary corrections.

[0605] A "generative AI model" is an algorithm or system that uses artificial intelligence to generate customized financial advice.

[0606] "Customized financial advice" is personalized advice created based on a user's individual financial data.

[0607] A "user terminal" is a device through which a user inputs financial data and receives financial advice sent from the server.

[0608] A "prompt" is a string of text data that is input into a generative AI model, and the AI ​​uses this to determine the output it generates.

[0609] The present invention is a system that allows users to input their own financial data and uses a generative AI model to provide customized financial advice. The following describes the processing for each server, terminal, and user in detail.

[0610] Server Processing

[0611] The server first receives financial data (income, expenses, asset status, risk tolerance, etc.) sent by the user. This data is received in real time and stored in an internal database, for example, using a relational database such as MySQL or PostgreSQL.

[0612] The server then uses external APIs to retrieve additional financial data, such as the user's bank account information and investment portfolio, using common APIs that are widely used for financial information.

[0613] All acquired data is consolidated and processed to remove missing or outlier values. Data cleansing is performed using Python or R libraries (such as Pandas). Once the data is clean and standardized, it is input into a generative AI model. This generative AI model is a natural language generation model, such as GPT-3 or BERT, which has been trained on past market trends and data from other users. Based on this, it generates optimal financial advice for the user. The generated advice is then formatted in text or graph form, visualized using tools such as Matplotlib, and sent back to the user's device.

[0614] Terminal handling

[0615] The terminal provides an interface for user input. The user uses a form to enter their income, expenses, and asset information, which is then sent to the server via the terminal. The entered data is sent to the server in real time using AJAX requests or WebSockets.

[0616] As financial advice is sent from the server, the device receives it in real time and updates its internal data model. It builds a dashboard using front-end frameworks such as React or Angular to display the advice in a format that's easy for users to understand. It also provides tools for users to take specific actions, such as setting savings goals or making investments. This includes interfaces for taking actions directly using APIs such as Twilio and Stripe.

[0617] User operations

[0618] Users first input their financial data into the device, including income, expenses, asset status, and risk tolerance, and then send this data to the server. Once the server sends them customized financial advice, the user can review it and decide on specific actions to apply.

[0619] For example, users can set monthly savings amounts based on a proposed savings plan, or purchase specific investment products according to a recommended investment strategy. Users can easily manage these actions on their device and track their progress in real time. This includes tracking user behavior using analytics tools such as Google Analytics and Mixpanel.

[0620] Specific examples

[0621] For example, User A has an annual income of 6 million yen, monthly living expenses of 200,000 yen, and a moderate risk tolerance. User A enters this data into a device and sends it to the server. The server receives the data and uses a generative AI model to generate an optimal savings and investment strategy for User A. The advice will be specific, such as "save 1 million yen per year and invest 500,000 yen in a risk-diversified fund."

[0622] An example of a prompt sentence to input to the generative AI model is as follows:

[0623] "User A's annual income is 6 million yen, and his monthly living expenses are 200,000 yen. User A has a medium risk tolerance. Based on this, please suggest the optimal savings and investment strategy."

[0624] This prompt is then fed into a generative AI model to generate specific financial advice for the user, helping them to manage their finances more efficiently.

[0625] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0626] Step 1: User enters financial data

[0627] The user enters their income, expenses, asset status, and risk tolerance into a form on the device, using text fields, drop-down lists, check boxes, etc.

[0628] input:

[0629] Income: 6 million yen

[0630] Expenses: 200,000 yen per month

[0631] Asset status: 5 million yen

[0632] Risk tolerance: Medium

[0633] output:

[0634] This financial data is entered into the terminal and converted into JSON format.

[0635] Specific behavior:

[0636] When the user clicks the "Submit" button on the form, the data is packaged in JSON format.

[0637] Step 2: Sending and Receiving Data

[0638] The terminal sends the entered financial data to the server, which receives it and stores it in a database. Real-time transmission is achieved using AJAX requests and WebSockets.

[0639] input:

[0640] Financial data in JSON format: {Annual income: $6,000,000, Monthly living expenses: $2,000, Asset status: $5,000, Risk tolerance: Medium}

[0641] output:

[0642] The server stores the data in a database.

[0643] Specific behavior:

[0644] The terminal sends JSON data, which the server receives and stores in a MySQL database.

[0645] Step 3: Obtaining additional data via an external API

[0646] The server uses an external API to retrieve the user's bank account information and investment portfolio data. It uses a financial information API (e.g., Plaid).

[0647] input:

[0648] User credentials

[0649] output:

[0650] Additional financial data (bank account balances, investment portfolios, etc.)

[0651] Specific behavior:

[0652] The server sends a request to the external API and stores the retrieved data back in the database.

[0653] Step 4: Integrate and cleanse the data

[0654] The server integrates all data it has acquired and cleanses missing and outlier values. It uses the Python Pandas library.

[0655] input:

[0656] Financial Data and Additional Financial Data

[0657] output:

[0658] Clean and standardized datasets

[0659] Specific behavior:

[0660] Run the Python script to load the data into a data frame and impute missing values.

[0661] Step 5: Generative AI model generates financial advice

[0662] The server inputs the cleansed data into a generative AI model, such as GPT-3, to generate optimal financial advice.

[0663] input:

[0664] Clean and standardized datasets

[0665] output:

[0666] Generated financial advice (e.g., "Save 1 million yen per year and invest 500,000 yen in a risk-diversified fund")

[0667] Specific behavior:

[0668] Prompt sentences are generated and input into a generative AI model to generate advice.

[0669] Step 6: Submitting and viewing advice

[0670] The server sends the generated financial advice to the user's device, which receives it and displays it on a dashboard. This uses React and Angular, among other tools.

[0671] input:

[0672] Generated Financial Advice

[0673] output:

[0674] Personalized financial advice displayed in a dashboard

[0675] Specific behavior:

[0676] The server sends advice in JSON format, which is displayed on the device's dashboard.

[0677] Step 7: Performing User Actions

[0678] Users can set and take specific actions based on the financial advice they receive, such as setting savings goals or making investments.

[0679] input:

[0680] Actions taken by the user

[0681] output:

[0682] The action taken and its result

[0683] Specific behavior:

[0684] Actions are executed when the user clicks on the dashboard's "Set Savings Goals" or "Invest" buttons. In some cases, Twilio or Stripe APIs are used.

[0685] (Application example 1)

[0686] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0687] In conventional financial management systems, it was difficult for users to input financial data and receive appropriate financial advice based on that data. Furthermore, the lack of means for users to set individual savings goals or run investment simulations made it difficult to provide optimal financial management for each individual user. The present invention aims to solve these problems and provide a system that allows users to easily and effectively manage their finances.

[0688] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0689] In this invention, the server includes means for inputting financial data from a user, means for transmitting the financial data to the server, means for using a generative AI model to analyze the financial data, means for generating customized financial advice using the generative AI model, means for transmitting the customized financial advice to a user terminal, means for displaying the customized financial advice on the user terminal, and means for setting savings goals and executing investment simulations on the user terminal, thereby enabling users to intuitively operate the system, receive customized financial advice, and effectively manage their savings and investments.

[0690] "User" means an individual or legal entity that uses the system, inputs financial data, and receives advice.

[0691] "Financial Data" refers to financial information such as a user's income, expenses, asset status, and risk tolerance.

[0692] "Server" means a computer system that receives financial data submitted by users, utilizes external APIs to obtain additional financial data, analyzes the data, and generates advice.

[0693] A "generative AI model" is a machine learning algorithm that analyzes a user's financial data and generates customized financial advice.

[0694] "Customized financial advice" refers to personalized investment strategies and savings plans created by generative AI models based on a user's financial data.

[0695] "User Device" means the device (e.g., smartphone, tablet, PC, etc.) used by a User to input financial data and view and interact with customized financial advice.

[0696] "External API" means an application program interface used by the Server to obtain additional financial data from external financial institutions or market data providers.

[0697] A "savings goal" refers to a specific savings amount and savings period set by a user, and is a numerical target that the user should aim for financially.

[0698] "Investment simulation" is a process of estimating future returns and risks in advance based on the amount of investment expected by the user.

[0699] "Data analysis" refers to the process of cleaning up financial and external data collected by the server, standardizing it, and inputting it into a generative AI model.

[0700] This invention is a system that allows users to input financial data and uses a generative AI model based on that data to provide customized financial advice. Below, we will explain in detail the processing for each server, terminal, and user.

[0701] Server Processing

[0702] The server receives financial data (income, expenses, asset status, risk tolerance, etc.) sent by the user. This data is received in real time and stored in an internal database. The server then uses external APIs to obtain additional financial data about the user's bank account information and investment portfolio. All of the obtained data is consolidated and cleansed of missing values ​​and outliers. Once the data is clean and standardized, it is input into a generative AI model. The generative AI model has learned from past market trends and data from other users and generates optimal financial advice for the user based on this. The generated advice is formatted in text and graphs and sent back to the user's device.

[0703] Terminal handling

[0704] The terminal provides an interface for user input. The user uses a form to enter their income, expenses, and asset information, which is then sent via the terminal to the server. The entered data sends requests and establishes communication with the server as needed. As financial advice is sent from the server, the terminal receives it in real time and updates its internal data model. It not only displays the advice in a user-friendly format (dashboards and reports), but also provides tools for the user to take specific actions (e.g., setting savings goals or simulating investments).

[0705] User operations

[0706] Users first enter their financial data into the device, including income, expenses, asset status, and risk tolerance, and then send this data to the server. After receiving customized financial advice from the server, users review it and decide on specific actions to apply. For example, they can set a monthly savings amount based on the proposed savings plan, or purchase specific investment products according to the recommended investment strategy. Users can easily manage these actions on the device and track their progress in real time.

[0707] Specific examples

[0708] For example, User A has an annual income of 6 million yen, monthly living expenses of 200,000 yen, and a moderate risk tolerance. User A enters this data into his device and sends it to the server. The server receives the data and uses a generative AI model to generate an optimal savings and investment strategy for User A. Specific advice is generated, such as "save 1 million yen per year and invest 500,000 yen in a risk-diversified fund." The server then sends this advice to User A's device. User A checks the advice on his device, sets actual savings goals through the app, and makes investments. This allows User A to efficiently manage his finances.

[0709] Hardware and software used

[0710] Hardware: Smartphones, tablets, computers

[0711] Software: Python, external APIs (e.g., APIs for retrieving financial data), generative AI models (e.g., GPT-4)

[0712] Prompt Sentence Examples

[0713] "User income: 6 million yen, monthly living expenses: 200,000 yen, assets: 1 million yen, risk tolerance: medium. Based on these criteria, please suggest the optimal savings plan and investment strategy for the user."

[0714] In this manner, the present invention is a system that enables users to receive and implement effective, customized financial advice.

[0715] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0716] Step 1:

[0717] The user enters their financial data into the terminal, including income, expenses, assets, risk tolerance, etc. The data from the user input form is then ready to be sent to the server.

[0718] Input: User's income, expenses, financial situation, risk tolerance

[0719] Output: Financial data entered into the terminal

[0720] Step 2:

[0721] The terminal sends the user's input data to the server, which receives the data in real time and stores it in an internal database.

[0722] Input: Financial data entered into the terminal

[0723] Output: Financial data stored on the server

[0724] Step 3:

[0725] The server uses external APIs to retrieve additional financial data about the user, such as bank account details and investment portfolio, which is then integrated with the existing user data.

[0726] Input: Financial data stored on the server, plus additional financial data retrieved from external APIs

[0727] Output: Consolidated comprehensive user financial data

[0728] Step 4:

[0729] The server cleanses the consolidated financial data and corrects missing and outlier values, ensuring clean and standardized input data for generative AI models.

[0730] Input: Integrated comprehensive user financial data

[0731] Output: Clean, cleaned financial data

[0732] Step 5:

[0733] The server then inputs the cleansed data into a generative AI model, which has learned about past market trends and data from other users, and generates optimal financial advice for the user based on this data.

[0734] Input: Clean, cleaned financial data

[0735] Output: Customized financial advice from a generative AI model

[0736] Step 6:

[0737] The server then formats the generated financial advice in text and graph format and sends it back to the user's terminal, which receives it in real time and displays it in a format that is easy for the user to understand.

[0738] Input: Customized financial advice from a generative AI model

[0739] Output: Financial advice displayed on the terminal

[0740] Step 7:

[0741] The terminal provides tools for users to set savings goals and run investment simulations. Users can set specific savings goals and investment simulations and execute plans based on them.

[0742] Input: Financial advice displayed on user terminal

[0743] Output: Savings goals and investment simulation plans set by the user

[0744] Step 8:

[0745] Users can access financial advice and take concrete actions on savings and investments through their devices, and the app tracks progress in real time, enabling them to effectively manage their finances.

[0746] Input: Savings goals and investment simulation plans set by the user

[0747] Output: Administrative data on savings and investment actions taken

[0748] Through these steps, the present invention enables users to receive and implement effective, customized financial advice.

[0749] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[0750] This invention is a system that allows users to input financial data and emotional data, and then provides customized financial advice based on that data using a generative AI model and an emotional engine. The following describes the processing for each server, terminal, and user in detail.

[0751] Server Processing

[0752] 1. Data reception:

[0753] The server first receives financial data (income, expenditure, asset status, risk tolerance, etc.) and emotional data (text messages, voice data, facial expression data, etc.) sent from the user's device. This data is sent to the server in real time and stored in an internal database.

[0754] 2. Data Integration:

[0755] The server uses external APIs to retrieve additional financial data about the user's bank account information and investment portfolio, and then integrates it with the received financial data. After data integration, it performs a cleansing process to correct missing values ​​and outliers.

[0756] 3. Emotion analysis:

[0757] The server uses an emotion engine to analyze the received emotion data, which identifies the user's emotional state (e.g., stress, relief, interest, etc.) and stores it in a database.

[0758] 4. Data Normalization and Analysis:

[0759] The cleansed data is standardized and fed into a generative AI model that takes into account market trends and past user data to generate optimal savings plans and investment strategies for users based on financial and sentiment analysis data.

[0760] 5. Advice generation and transmission:

[0761] The generated advice is formatted as text and graphs and sent to the user's device in an appropriate communication format (e.g., encouraging words or detailed explanations) based on the results of sentiment analysis.

[0762] Terminal handling

[0763] 1. Data Entry:

[0764] The terminal provides a form for users to input financial data and emotional data, including income, expenses, asset information, and risk tolerance, and emotional data including text messages, voice data, and facial expression data.

[0765] 2. Real-time validation and submission:

[0766] Once financial and emotional data is entered, the terminal validates it in real time, prompting the user to re-enter any incomplete data, and then sends the validated data to the server.

[0767] 3. Advice Receiving and Display:

[0768] The terminal receives customized financial advice sent from the server and converts it into an internal data model. The terminal visualizes the received advice and displays it in the form of a dashboard or report. Based on the results of sentiment analysis, the terminal provides advice to the user in an appropriate tone and format.

[0769] 4. Take action:

[0770] The device provides an interface for users to take specific actions based on the advice (such as setting savings goals or making investments), and the progress of the actions taken by the user is updated in real time on the device.

[0771] User operations

[0772] 1. Data Entry:

[0773] Users access the terminal and input their financial data (income, expenses, asset status, risk tolerance) and emotional data (text, voice, facial expressions).

[0774] 2. Data transmission:

[0775] Once the input is complete, the user presses the submit button to send the data to the server, and a notification message is displayed when the data is sent.

[0776] 3. Advice confirmation:

[0777] The user checks the customized financial advice sent from the server along with additional information based on sentiment analysis. For example, if the advice is to "save 1 million yen a year and invest 500,000 yen in a risk-diversified fund," an encouraging message and explanation of the risks will be added based on the results of sentiment analysis.

[0778] 4. Action execution:

[0779] Users can then take specific financial actions based on the advice, such as setting up regular savings or purchasing investment products, via their device.

[0780] 5. Feedback and Replanning:

[0781] The user frequently checks the progress of the action on the device, replans if necessary, and receives feedback based on their emotional state to help generate advice for the next action.

[0782] Specific examples

[0783] For example, User B has an annual income of 8 million yen, monthly living expenses of 250,000 yen, and a high risk tolerance. User B enters this financial data and emotional data, such as "I've been feeling stressed lately," into his / her device and sends it to the server. The server receives this data and analyzes it using a generative AI model and an emotion engine. As a result, it generates advice that is optimal for User B, such as "invest 1.5 million yen per year in a risk-diversified fund and put 500,000 yen into short-term savings." Furthermore, based on the results of the emotion analysis, it also provides "suggestions for low-risk investment strategies to reduce stress" and "detailed explanations to give a sense of security about investing."

[0784] In this way, the present invention provides customized financial advice that takes into account a user's overall financial situation and emotional state, helping users manage their finances more effectively.

[0785] The processing flow will be explained below.

[0786] Server processing flow

[0787] Step 1:

[0788] The server receives financial data and emotional data in real time from the user terminal. The financial data includes income, expenditure, asset status, and risk tolerance, and the emotional data includes text messages, voice data, and facial expression data.

[0789] Step 2:

[0790] The server uses external APIs to obtain additional financial data about the user's bank account details and investment portfolio, which is then stored in an internal database.

[0791] Step 3:

[0792] The server integrates the received financial data with the additional financial data it acquires, and performs a cleansing process to correct missing or outlier values.

[0793] Step 4:

[0794] The server uses an emotion engine to analyze the received emotion data, extracting emotion tags (e.g., stress, relief, interest, etc.) from text messages and voice data.

[0795] Step 5:

[0796] The server standardizes the cleansed financial data and feeds it, along with sentiment analysis results, into the generative AI model. The standardization process includes scaling numerical data and encoding categorical data.

[0797] Step 6:

[0798] The generative AI model analyzes the input data and generates optimal savings plans and investment strategies for users, and the analysis results are converted into text and graph formats.

[0799] Step 7:

[0800] The server then adds a message with an appropriate tone based on the results of sentiment analysis to the generated advice. For example, if a user is under stress, the server might recommend "low-risk investments" and add words of reassurance.

[0801] Step 8:

[0802] The server then sends the prepared advice to the user's device, securely using the HTTPS protocol.

[0803] Terminal processing flow

[0804] Step 1:

[0805] The terminal displays a form for users to enter financial and emotional data, including income, expenses, asset information, risk tolerance, and emotional data (text messages, voice, and facial expressions).

[0806] Step 2:

[0807] As users enter data into forms, the terminal validates the input data in real time and prompts the user to re-enter the data if it is incomplete.

[0808] Step 3:

[0809] The validated data is sent to the server securely using the HTTPS protocol.

[0810] Step 4:

[0811] The terminal receives customized financial advice sent from the server and converts the received data into an internal data model.

[0812] Step 5:

[0813] The device visualizes and displays advice to the user in the form of a dashboard or report, and delivers messages in an appropriate tone based on the results of sentiment analysis.

[0814] Step 6:

[0815] Providing an interface for users to take specific actions based on advice, including interfaces to help set savings goals and make investments.

[0816] Step 7:

[0817] The progress of actions performed by the user is updated in real time and displayed on the device.

[0818] User Process Flow

[0819] Step 1:

[0820] Users access the terminal and input their financial data (income, expenses, asset status, risk tolerance) and emotional data (text messages, voice, facial expressions).

[0821] Step 2:

[0822] Once the input is complete, the user presses the submit button to send the data to the server, and a notification message is displayed when the data has been sent.

[0823] Step 3:

[0824] The user checks the customized financial advice sent from the server and messages based on sentiment analysis. For example, the advice might be "save 1 million yen a year and invest 500,000 yen in a risk-diversified fund," along with suggestions for low-risk investment strategies to reduce stress and encouraging messages.

[0825] Step 4:

[0826] Users take specific financial actions based on the advice, such as setting up regular savings or purchasing investment products.

[0827] Step 5:

[0828] Users frequently check the progress of their actions on their devices and make replanning and adjustments as needed.

[0829] Step 6:

[0830] Users can provide feedback on their achieved goals and ongoing plans via their devices, which will be used to generate the next advice. This feedback will be used to provide even more accurate advice.

[0831] Example 2

[0832] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0833] In modern financial management, not only individual users' financial data but also their emotional state at any given time is an important factor. However, conventional financial advisory systems have been unable to fully utilize emotional data, making it difficult to provide customized advice. This has led to issues such as users feeling stressed when making appropriate financial management and investment decisions, making it difficult to manage their finances efficiently.

[0834] The identification process by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means. In this invention, the server includes: means for inputting financial data and emotion data from a user; means for transmitting the financial data and emotion data to the server; means for using a generative AI model and an emotion engine to analyze the data; means for generating customized financial advice using the generative AI model and emotion engine; means for transmitting the customized financial advice to a user terminal; means for displaying the customized financial advice on the user terminal; means for the server to obtain additional financial data using an external API and integrate the data; means for the server to analyze the emotion data and identify the user's emotional state; means for the user terminal to validate the financial data and emotion data in real time and notify the user of any input errors; means for the customized financial advice to be provided to the user in a form that reflects the emotion analysis results; means for the user terminal to support the user in taking action based on the advice; and means for the user to provide feedback that is used to generate the next advice. This enables more personalized financial management for users and provides optimal advice that takes into account their emotional state and market trends.

[0835] markdown

[0836] A "user" is an individual or entity that uses the system to input financial and emotional data and receive customized financial advice.

[0837] "Financial data" refers to data relating to the user's economic situation, such as income, expenses, asset status, and risk tolerance.

[0838] "Emotion data" refers to data such as text messages, voice data, and facial expression data that indicate the user's emotional state.

[0839] A "server" is a central computing unit that receives data sent from user terminals, performs analysis, and generates financial advice.

[0840] A "generative AI model" is a machine learning model that takes into account market trends and historical data to generate customized financial advice based on input data.

[0841] An "emotion engine" is a software module for analyzing emotion data and identifying the user's emotional state.

[0842] A "user terminal" is an electronic device through which a user inputs financial and emotional data and receives customized financial advice transmitted from a server.

[0843] An "external API" is an interface for obtaining data from other systems or services.

[0844] "Real-time validation" is a process that immediately checks input data and prompts correction of any incomplete data.

[0845] "Customized financial advice" is advice generated by a generative AI model and emotion engine based on a user's individual financial situation and emotional state.

[0846] A "means for supporting behavioral execution" is a part of the system that provides an interface or functionality for users to take specific actions based on customized financial advice.

[0847] "Feedback" refers to opinions and evaluations provided by users to the system, and is information that will be used to generate the next piece of advice.

[0848] MODE FOR CARRYING OUT THE INVENTION

[0849] The present invention is a system for providing optimal financial advice to users based on financial data and emotional data. The processing performed by the server, terminal, and user will be described in detail below.

[0850] Server Processing

[0851] Data reception

[0852] The server first receives financial data (income, expenses, asset status, risk tolerance, etc.) and emotional data (text messages, voice data, facial expression data, etc.) sent from the user's device. This data is sent to the server in real time and stored in an internal database. For example, if a user sends data from their device such as "monthly income of 500,000 yen, monthly expenses of 300,000 yen," this information is stored in the receiving buffer.

[0853] Data Integration

[0854] The server uses external APIs to retrieve additional financial data about the user's bank account information and investment portfolio, and then integrates it with the original financial data. This process also includes a cleansing process to automatically correct missing data and outliers. For example, if a user has multiple accounts and their account information is retrieved via an API, the collected data can be integrated with the input data.

[0855] Emotion analysis

[0856] The server uses an emotion engine to analyze the received emotion data. Through this analysis, the user's emotional state (e.g., stress, relief, interest, etc.) is identified and stored in a database. For example, from a user's message such as "I'm tired today," the emotion engine determines the user's stress level.

[0857] Data normalization and analysis

[0858] The cleansed data is then standardized and fed into a generative AI model, which takes into account market trends and historical data to generate optimal savings plans and investment strategies for users based on financial and sentiment data. For example, historical market data could be used to suggest optimal investment options for users.

[0859] Advice generation and delivery

[0860] The generated advice is then formatted as text and graphs and sent to the user in an appropriate communication format based on the results of the emotion analysis. For example, advice containing encouraging words and detailed explanations may be sent.

[0861] Terminal handling

[0862] Data Entry

[0863] The device provides a form for users to input financial and emotional data. Financial data includes income, expenses, asset information, and risk tolerance, while emotional data includes text messages, voice data, and facial expression data. For example, the device has text boxes for inputting "income" and "expenses," as well as voice recording and facial expression recognition functions for expressing "emotions."

[0864] Real-time validation and submission

[0865] When financial and emotional data is entered, the terminal validates it in real time and displays a warning if the data is incomplete. For example, if a negative income value is entered, an error message is displayed prompting the user to correct it. The terminal also sends the validated data to the server.

[0866] Advice reception and display

[0867] The device receives customized financial advice sent from the server and displays it visually, including graphical dashboards and detailed text messages. For example, if the advice is to "save 1 million yen a year and invest 500,000 yen in a risk-diversified fund," it will visualize it in pie charts and bar graphs.

[0868] Action Execution

[0869] The device provides an interface for users to take action based on the advice. For example, a "Set Savings Goal" button or an "Invest" button are displayed, and users can tap the button to start taking specific action.

[0870] User operations

[0871] Data Entry

[0872] The user uses the device to input their own financial data (income, expenditure, asset status, risk tolerance) and emotional data (text, voice, facial expression). For example, they input "income 500,000 yen, expenditure 300,000 yen" and also emotional data such as "I've been under a lot of stress lately."

[0873] Data transmission

[0874] The data you have entered will be sent to the server. When you press the send button, the progress will be displayed and you will be notified when the data has been sent.

[0875] Advice confirmation

[0876] Check the customized advice sent from the server. For example, "Invest 1.5 million yen a year in a risk-diversified fund and put 500,000 yen into short-term savings," along with an encouraging message based on the results of sentiment analysis.

[0877] Action execution

[0878] The user then takes specific actions based on the advice, such as purchasing an investment fund, and checks the progress of the action on the device.

[0879] Feedback and replanning

[0880] Users can track the progress of their actions, reorganize their plans if necessary, and provide feedback on their emotional state to help inform the next recommendation. For example, they can provide feedback like, "I'm happy with my current investments, but I'd like to see a less risky option," which will be reflected in the next recommendation.

[0881] Specific examples

[0882] For example, User B has an annual income of 8 million yen, monthly living expenses of 250,000 yen, and a high risk tolerance. User B enters this financial data and emotional data, such as "I've been feeling stressed lately," into his / her device and sends it to the server. The server analyzes the received data and generates advice such as "invest 1.5 million yen per year in a risk-diversified fund and put the remaining 500,000 yen into short-term savings." Furthermore, based on the results of the emotional analysis, the advice is accompanied by "low-risk investment strategies to reduce stress" and "detailed explanations to give a sense of security about investing." This advice is displayed on the device, and the user then takes specific action (purchase an investment fund) through the device.

[0883] Prompt Sentence Examples

[0884] "Enter information about your income and expenses, financial situation, and risk tolerance, and provide your recent emotional state via text, voice, and facial expressions."

[0885] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0886] System program processing flow

[0887] Server Processing

[0888] Step 1: Receiving data

[0889] The server receives financial data (income, expenditure, asset status, risk tolerance) and emotional data (text messages, voice data, facial expression data) sent from the user's device. This data is stored in a receiving buffer in real time. For example, information entered by a user as income of 500,000 yen and expenditure of 300,000 yen is saved.

[0890] Input: Financial and emotional data from user devices

[0891] Output: Data stored in the receive buffer

[0892]

[0893] Step 2: Data Integration

[0894] The server uses external APIs to retrieve additional financial data about the user's bank account information and investment portfolio, and combines it with the incoming financial data. This combined data undergoes cleansing processes to correct missing and outlier values. For example, data retrieved from financial institution APIs can be combined with user input data.

[0895] Input: Incoming financial data and financial data obtained from external APIs

[0896] Output: Unified and cleansed data

[0897]

[0898] Step 3: Sentiment Analysis

[0899] The server uses an emotion engine to analyze the received emotion data. This analysis identifies the user's emotional state (e.g., stress, relief, interest) and stores it in a database. For example, if a user says, "I'm tired today," the emotion engine analyzes the user's stress level.

[0900] Input: Received emotion data

[0901] Output: Parsed emotional state data

[0902]

[0903] Step 4: Data standardization and analysis

[0904] The server standardizes the cleansed data and inputs it into a generative AI model, which takes into account market trends and historical data, and generates optimal savings plans and investment strategies for users based on financial and sentiment data. For example, it might suggest optimal investments based on user data combined with market trend data.

[0905] Input: Cleansed and standardized data

[0906] Output: Analysis results from the generative AI model (savings plan and investment strategy)

[0907]

[0908] Step 5: Advice generation and sending

[0909] The generated advice is then formatted as text and graphs and sent to the user in an appropriate communication format based on the results of the emotion analysis. For example, advice containing encouraging words and detailed explanations may be sent.

[0910] Input: Analysis results and emotional state data

[0911] Output: Customized financial advice sent to the user

[0912] Terminal handling

[0913] Step 1: Data entry

[0914] The device provides a form for users to enter financial and emotional data, including text boxes for entering income and expenses, as well as voice recording and facial recognition capabilities.

[0915] Input: User financial and emotional data input

[0916] Output: Data entered in the input form

[0917]

[0918] Step 2: Real-time validation and submission

[0919] The terminal validates the data entered in real time and displays a warning if the data is incomplete. For example, if a negative income value is entered, an error message is displayed and the user is prompted to correct the data. After validation is complete, the data is sent to the server.

[0920] Input: User-entered financial and sentiment data

[0921] Output: Send validated data to the server

[0922]

[0923] Step 3: Receive and view advice

[0924] The terminal receives customized financial advice sent from the server and displays it in a visualized format (dashboard and graphs). For example, advice such as "save 1 million yen a year and invest 500,000 yen in a risk-diversified fund" is visualized.

[0925] Input: Customized financial advice from the server

[0926] Output: Visualized advice display

[0927]

[0928] Step 4: Take Action

[0929] The device provides an interface for users to take specific actions based on the advice, such as a "Set Savings Goal" button or an "Invest" button that users can tap to start the action.

[0930] Enter: customized financial advice

[0931] Output: User action execution

[0932] User operations

[0933] Step 1: Data entry

[0934] The user inputs financial data (income, expenditure, asset status, risk tolerance) and emotional data (text, voice, facial expression) into the terminal. For example, the user inputs "income 500,000 yen, expenditure 300,000 yen" and emotional data such as "I've been under a lot of stress lately."

[0935] Input: Financial and sentiment data

[0936] Output: Filling in the input form

[0937]

[0938] Step 2: Send data

[0939] The data you have entered will be sent to the server. When you press the send button, the progress will be displayed and you will be notified when the data has been sent.

[0940] Input: Data entered into an input form

[0941] Output: Send data to the server

[0942]

[0943] Step 3: Check the advice

[0944] The user then checks the customized financial advice sent from the server, such as "Invest 1.5 million yen a year in a risk-diversified fund and put 500,000 yen into short-term savings," along with an encouraging message based on the results of sentiment analysis.

[0945] Input: Customized financial advice sent from the server

[0946] Output: Check the visualized advice

[0947]

[0948] Step 4: Take action

[0949] The user then takes specific actions based on the advice, such as purchasing an investment fund, and checks the progress of the action on the device.

[0950] Enter: customized financial advice

[0951] Output: Implementing specific actions

[0952]

[0953] Step 5: Feedback and replanning

[0954] Users can track their progress and adjust their plans as needed. The system also provides feedback based on their emotional state, such as, "I'm happy with my current investments, but I'd like to see a less risky option."

[0955] Input: Action execution results and emotional feedback

[0956] Output: Data used to generate next advice

[0957]

[0958] (Application example 2)

[0959] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0960] In conventional factory work, workers' emotional states have a significant impact on work performance and safety measures, but there is a lack of effective ways to manage them in real time. Furthermore, there is a lack of technology to provide optimal advice to each individual worker. Therefore, there is a need to improve work performance and strengthen safety measures.

[0961] The specific processing by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for inputting financial data and emotional data from a user, means for transmitting the financial data and emotional data to the server, means for using a generative AI model and an emotion engine that analyzes the financial data and emotional data, means for generating customized financial advice and work performance improvement advice using the generative AI model and emotion engine, means for transmitting the customized financial advice and work performance improvement advice to a user terminal, and means for displaying the customized financial advice and work performance improvement advice on the user terminal. This makes it possible to analyze the emotional state of a worker in real time and provide optimal advice.

[0962] "User" refers to a person or organization that uses the system.

[0963] "Financial Data" refers to financial information about an individual or organization, such as income, expenses, assets, and risk tolerance.

[0964] "Emotional data" refers to information that represents an emotional state, such as text messages, voice data, and facial expression data.

[0965] "Server" refers to the central processing unit that receives and analyzes data sent by the User and retransmits the generated information to the User.

[0966] "Generative AI model" refers to a machine learning model that analyzes a user's financial and emotional data to generate customized advice.

[0967] "Emotion engine" refers to an analysis system for analyzing emotion data and identifying a user's emotional state.

[0968] "User terminal" refers to a device used by a user to input data or receive and display information sent from the server.

[0969] "External Data Acquisition Means" refers to means for acquiring and integrating additional information from external data sources.

[0970] "Work data" refers to data related to the progress and performance of work in a factory or on-site.

[0971] "Operation data" refers to data relating to the operating status of machinery and equipment within a factory.

[0972] The present invention is a system that utilizes a user's financial data and emotion data to provide customized financial advice, including work performance improvement and safety measures, using a generative AI model and emotion engine. Specific processing for each server, terminal, and user is described below.

[0973] Server Processing

[0974] 1. Data Reception

[0975] The server receives financial data (e.g., income, expenses, asset status, risk tolerance) and emotional data (e.g., text messages, voice data, facial expression data) sent from the user's device. This data is sent to the server in real time and stored in an internal database.

[0976] 2. External Data Acquisition and Data Integration

[0977] The server uses external data acquisition means to acquire work data and equipment operation data. The acquired data is integrated and cleansed. This process uses data integration programs and data cleaning algorithms in a Python environment.

[0978] 3. Sentiment Analysis and Data Analysis

[0979] The server analyzes the emotion data using an emotion engine (e.g., Spacy, IBM Watson). The analysis results are combined with financial data and input into a generative AI model (e.g., TensorFlow, PyTorch). The generative AI model uses this data to generate customized financial advice and work performance improvement advice.

[0980] 4. Advice Generation and Delivery

[0981] The generated advice is formatted as text and graphs and sent to the user's device in an appropriate communication format (e.g., detailed explanations that provide a sense of security) based on the results of sentiment analysis.

[0982] Terminal handling

[0983] 1. Data Entry

[0984] The device provides an interface for users to input financial and emotional data, including text fields, voice input, and facial expression input using a camera.

[0985] 2. Data Transmission

[0986] The entered data is sent from the terminal to the server. This process includes a data transmission program and validation check function.

[0987] 3. Receiving and displaying advice

[0988] The customized advice sent from the server is received by the device, visualized, and displayed to the user. This user interface includes dashboards and reports. In particular, the advice for improving work performance is displayed on smart glasses.

[0989] User operations

[0990] 1. Data Entry

[0991] The user accesses the input interface of the terminal and inputs their own financial data (e.g., income, expenses, asset status, risk tolerance) and emotional data (e.g., text, voice, facial expression).

[0992] 2. Data Transmission

[0993] Once the input is complete, the user presses the submit button to send the data to the server, and a notification message is displayed when the data is sent.

[0994] 3. Check and implement the advice

[0995] After checking the customized advice sent from the server, the user can take specific actions based on the advice, for example, a worker can adjust their work performance or take a break according to the advice displayed on the smart glasses.

[0996] Specific examples

[0997] For example, if Worker A working in a factory comments that he has been under a lot of stress lately, and this emotional state is also confirmed by facial expression data, the server will generate advice to Worker A based on this, recommending that he prioritize low-risk work and take a temporary break. This advice is displayed on the smart glasses.

[0998] Prompt Sentence Examples

[0999] "Design a real-time emotion analysis application for factory workers. This application requires workers to input their emotion data (e.g., stress or relief) and work performance data, and then displays appropriate advice in real time on smart glasses. Data analysis is based on a generative AI model and an emotion engine. Please also generate the corresponding Python code."

[1000] The present invention is a system that handles emotional data and financial data in an integrated manner to provide customized advice to factory workers, thereby improving productivity and strengthening safety measures.

[1001] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1002] Step 1: The user inputs financial and emotional data into the terminal.

[1003] Users use the device's interface to input financial data such as income, expenses, asset status, and risk tolerance. They also input text messages, voice data, and facial expression data that indicate their emotional state via a camera or microphone. The input data is temporarily stored in the device.

[1004] Step 2: The terminal performs real-time validation and sends the data to the server.

[1005] The terminal validates the entered financial and emotional data in real time to check for incomplete data or outliers. Once validated, the data is encrypted and sent to the server, where it checks the input data (income, expenses, emotional state) and either displays an error message or converts it into a format that can be sent.

[1006] Step 3: The server receives the data and stores it in the database.

[1007] The server receives the financial and emotional data sent from the device and stores it in an internal database, which is used in subsequent processing steps.

[1008] Step 4: The server acquires the additional data using an external data acquisition means.

[1009] The server makes API calls to retrieve work data and equipment operation data from external sources. The retrieved data is integrated with internal data and stored in a database. Additional data (such as equipment operation status) is retrieved and integrated with the existing database.

[1010] Step 5: The server performs the cleansing process.

[1011] The server performs a cleansing process on all the acquired data, filling in missing values ​​and correcting outliers, using a Python data analysis library.

[1012] Step 6: The server performs sentiment analysis.

[1013] The server analyzes the emotional data using an emotion engine (e.g., Spacy, IBM Watson). The analysis results are stored in a database as the user's emotional state (stress, relief, etc.). Input: Emotional data (text, voice, facial expression), Output: Emotional state (stress, elation, etc.).

[1014] Step 7: The server inputs the data into the generative AI model and performs analysis.

[1015] The cleansed data and sentiment analysis results are input into a generative AI model. The generative AI model (e.g., TensorFlow, PyTorch) analyzes the financial data and integrated data to generate customized financial and work performance improvement advice. Input: Financial data, sentiment data, work data. Output: Customized advice (savings plans, investment strategies, work instructions, etc.).

[1016] Step 8: The server sends the generated advice.

[1017] The generated advice is formatted as text and graphs and sent to the user terminal in an appropriate communication format. Input: Advice data (text, graphs), Output: Transmission to the user terminal.

[1018] Step 9: The device receives and displays the advice.

[1019] The user device receives the customized advice sent from the server and visualizes and displays it. The display format is a dashboard or report. In particular, advice for improving work performance is displayed on smart glasses, etc. Input: Received data (advice), Output: Visual display (graph, text).

[1020] Step 10: The user acts on the advice.

[1021] The user checks the customized advice displayed on the device and takes specific actions based on it, such as creating a savings plan or changing the priorities of tasks. Input: Check the advice, Output: Specific actions (savings, investment, task change, etc.).

[1022] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1023] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1024] In the above embodiment, an example in which the specific processing is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the smart glasses 214.

[1025] [Third embodiment]

[1026] FIG. 5 shows an example of the configuration of a data processing system 310 according to the third embodiment.

[1027] 5, the data processing system 310 includes the data processing device 12 and a headset type terminal 314. An example of the data processing device 12 is a server.

[1028] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1029] The headset type terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a display 343. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the display 343 are also connected to the bus 52.

[1030] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1031] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1032] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1033] Fig. 6 shows an example of the main functions of the data processing device 12 and the headset type terminal 314. As shown in Fig. 6, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1034] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1035] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1036] In the headset type terminal 314, a reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1037] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the headset type terminal 314 will be referred to as the "terminal."

[1038] The present invention is a system that allows users to input financial data and uses a generative AI model based on that data to provide customized financial advice. The following describes the processing for each server, terminal, and user in detail.

[1039] Server Processing

[1040] The server first receives financial data (income, expenses, assets, risk tolerance, etc.) submitted by the user. This data is received in real time and stored in an internal database. The server then utilizes external APIs to obtain additional financial data about the user's bank account information and investment portfolio.

[1041] All acquired data is consolidated and cleansed of missing or outlier values. Once the data is clean and standardized, it is fed into a generative AI model. The generative AI model has learned from past market trends and data from other users, and uses this information to generate optimal financial advice for the user. The generated advice is then formatted in text and graphs and sent back to the user's device.

[1042] Terminal handling

[1043] The terminal provides an interface for user input. The user uses a form to enter their income, expenses, and asset information, which is then sent to the server via the terminal. The entered data sends requests as needed to establish communication with the server.

[1044] As financial advice is sent from the server, the device receives it in real time, updates its internal data model, and presents it in a user-friendly format (dashboards and reports). It also provides tools for users to take specific actions, such as setting savings goals or making investments.

[1045] User operations

[1046] Users first input their financial data into the device, including income, expenses, asset status, and risk tolerance, and then send this data to the server. Once the server sends them customized financial advice, the user can review it and decide on specific actions to apply.

[1047] For example, users can set monthly savings amounts based on a proposed savings plan, or purchase specific investment products according to a recommended investment strategy. Users can easily manage these actions on their device and track their progress in real time.

[1048] Specific examples

[1049] For example, User A has an annual income of 6 million yen, monthly living expenses of 200,000 yen, and a moderate risk tolerance. User A enters this data into a terminal and sends it to the server. The server receives the data and uses a generative AI model to generate an optimal savings and investment strategy for User A. Specific advice such as "save 1 million yen per year and invest 500,000 yen in a risk-diversified fund" is generated.

[1050] The server then sends this advice to User A's device. User A checks the advice on the device, sets savings goals through the app, and makes investments. This allows User A to efficiently manage their finances.

[1051] In this manner, the present invention is a system that enables users to receive and implement effective, customized financial advice.

[1052] The processing flow will be explained below.

[1053] Server processing flow

[1054] Step 1:

[1055] The server receives financial data sent from the user terminal, including income, expenses, asset status, risk tolerance, etc.

[1056] Step 2:

[1057] The server uses external APIs to obtain additional financial data about the user, such as bank account details and investment portfolios, and receives the response of the external API and stores it in an internal database.

[1058] Step 3:

[1059] The server integrates the received financial data with the additional financial data it has acquired. After the data integration, it performs a cleansing process to correct missing and outlier values.

[1060] Step 4:

[1061] The server standardizes the cleansed data and inputs it into the generative AI model. The standardization process includes scaling numerical data and encoding categorical data.

[1062] Step 5:

[1063] A generative AI model analyzes input data and generates optimal savings and investment strategies for users, taking into account historical market trends and data from other users.

[1064] Step 6:

[1065] The server converts the generated advice into text and graphical formats, making it easily understandable to the user.

[1066] Step 7:

[1067] The server then sends the prepared advice to the user's device, securely using the HTTPS protocol.

[1068] Terminal processing flow

[1069] Step 1:

[1070] The terminal displays a form for the user to enter financial data, including income, expenses, asset information, and risk tolerance.

[1071] Step 2:

[1072] As users enter data into forms, the terminal validates the input data in real time and prompts the user to re-enter any incomplete data.

[1073] Step 3:

[1074] The terminal sends the validated data to the server using the HTTPS protocol to ensure the safety of the user data.

[1075] Step 4:

[1076] When financial advice is sent from the server, the terminal receives it and converts the received data into an internal data model.

[1077] Step 5:

[1078] The device visualizes the advice it receives and displays it in the form of a dashboard or report that is easy for the user to understand.

[1079] Step 6:

[1080] The terminal provides an interface that allows users to take specific actions based on the advice (such as setting savings goals or making investments).

[1081] User Process Flow

[1082] Step 1:

[1083] The user accesses the terminal and inputs their income, expenses, asset status, and risk tolerance, entering the required information into the input form one by one.

[1084] Step 2:

[1085] Once the input is complete, the user presses the submit button to send the data to the server, and a notification message is displayed when the data is sent.

[1086] Step 3:

[1087] When the advice is sent from the server, the user checks it on the device, understands the advice, and prepares by taking notes of any necessary parts.

[1088] Step 4:

[1089] Users take specific financial actions based on the advice, such as setting up regular savings or purchasing investment products.

[1090] Step 5:

[1091] Users frequently check the progress of their actions on their devices and make replanning and adjustments as needed.

[1092] Step 6:

[1093] Users can provide feedback on their achieved goals and ongoing plans via the device, and ask for further advice. This feedback will help provide more accurate advice.

[1094] Example 1

[1095] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1096] Conventional financial advice systems face the problem of being unable to provide adequately customized advice based on a user's individual financial data. In particular, complex processes are required, such as real-time data integration and cleansing, and the acquisition of additional data using external APIs. However, because these processes are not sufficiently optimized, users are unable to receive highly accurate advice.

[1097] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1098] In this invention, the server includes means for inputting financial data from a user, means for transmitting the financial data to the server, means for the server to obtain additional financial data using an external API, means for integrating the financial data with the additional financial data and cleansing missing values ​​and outliers, means for generating customized financial advice using the generative AI model, means for transmitting the customized financial advice to a user terminal, and means for displaying the customized financial advice on the user terminal, thereby enabling more accurate customized financial advice to be provided to users in real time.

[1099] A "user" is a person or entity that inputs financial data and receives customized financial advice.

[1100] "Financial data" is a general term for information about a user's financial situation, such as income, expenses, asset status, and risk tolerance.

[1101] A "server" is a computer system for receiving and processing financial data submitted by users.

[1102] "External API" means an application program interface for accessing third-party services to obtain additional financial data.

[1103] "Financial data" is a general term for information about bank accounts and investment portfolios.

[1104] "Integration" refers to the process of bringing together data obtained from different data sources.

[1105] "Cleansing" is a data preparation process that removes missing and outliers from data and makes any necessary corrections.

[1106] A "generative AI model" is an algorithm or system that uses artificial intelligence to generate customized financial advice.

[1107] "Customized financial advice" is personalized advice created based on a user's individual financial data.

[1108] A "user terminal" is a device through which a user inputs financial data and receives financial advice sent from the server.

[1109] A "prompt" is a string of text data that is input into a generative AI model, and the AI ​​uses this to determine the output it generates.

[1110] The present invention is a system that allows users to input their own financial data and uses a generative AI model to provide customized financial advice. The following describes the processing for each server, terminal, and user in detail.

[1111] Server Processing

[1112] The server first receives financial data (income, expenses, asset status, risk tolerance, etc.) sent by the user. This data is received in real time and stored in an internal database, for example, using a relational database such as MySQL or PostgreSQL.

[1113] The server then uses external APIs to retrieve additional financial data, such as the user's bank account information and investment portfolio, using common APIs that are widely used for financial information.

[1114] All acquired data is consolidated and processed to remove missing or outlier values. Data cleansing is performed using Python or R libraries (such as Pandas). Once the data is clean and standardized, it is input into a generative AI model. This generative AI model is a natural language generation model, such as GPT-3 or BERT, which has been trained on past market trends and data from other users. Based on this, it generates optimal financial advice for the user. The generated advice is then formatted in text or graph form, visualized using tools such as Matplotlib, and sent back to the user's device.

[1115] Terminal handling

[1116] The terminal provides an interface for user input. The user uses a form to enter their income, expenses, and asset information, which is then sent to the server via the terminal. The entered data is sent to the server in real time using AJAX requests or WebSockets.

[1117] As financial advice is sent from the server, the device receives it in real time and updates its internal data model. It builds a dashboard using front-end frameworks such as React or Angular to display the advice in a format that's easy for users to understand. It also provides tools for users to take specific actions, such as setting savings goals or making investments. This includes interfaces for taking actions directly using APIs such as Twilio and Stripe.

[1118] User operations

[1119] Users first input their financial data into the device, including income, expenses, asset status, and risk tolerance, and then send this data to the server. Once the server sends them customized financial advice, the user can review it and decide on specific actions to apply.

[1120] For example, users can set monthly savings amounts based on a proposed savings plan, or purchase specific investment products according to a recommended investment strategy. Users can easily manage these actions on their device and track their progress in real time. This includes tracking user behavior using analytics tools such as Google Analytics and Mixpanel.

[1121] Specific examples

[1122] For example, User A has an annual income of 6 million yen, monthly living expenses of 200,000 yen, and a moderate risk tolerance. User A enters this data into a device and sends it to the server. The server receives the data and uses a generative AI model to generate an optimal savings and investment strategy for User A. The advice will be specific, such as "save 1 million yen per year and invest 500,000 yen in a risk-diversified fund."

[1123] An example of a prompt sentence to input to the generative AI model is as follows:

[1124] "User A's annual income is 6 million yen, and his monthly living expenses are 200,000 yen. User A has a medium risk tolerance. Based on this, please suggest the optimal savings and investment strategy."

[1125] This prompt is then fed into a generative AI model to generate specific financial advice for the user, helping them to manage their finances more efficiently.

[1126] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1127] Step 1: User enters financial data

[1128] The user enters their income, expenses, asset status, and risk tolerance into a form on the device, using text fields, drop-down lists, check boxes, etc.

[1129] input:

[1130] Income: 6 million yen

[1131] Expenses: 200,000 yen per month

[1132] Asset status: 5 million yen

[1133] Risk tolerance: Medium

[1134] output:

[1135] This financial data is entered into the terminal and converted into JSON format.

[1136] Specific behavior:

[1137] When the user clicks the "Submit" button on the form, the data is packaged in JSON format.

[1138] Step 2: Sending and Receiving Data

[1139] The terminal sends the entered financial data to the server, which receives it and stores it in a database. Real-time transmission is achieved using AJAX requests and WebSockets.

[1140] input:

[1141] Financial data in JSON format: {Annual income: $6,000,000, Monthly living expenses: $2,000, Asset status: $5,000, Risk tolerance: Medium}

[1142] output:

[1143] The server stores the data in a database.

[1144] Specific behavior:

[1145] The terminal sends JSON data, which the server receives and stores in a MySQL database.

[1146] Step 3: Obtaining additional data via an external API

[1147] The server uses an external API to retrieve the user's bank account information and investment portfolio data. It uses a financial information API (e.g., Plaid).

[1148] input:

[1149] User credentials

[1150] output:

[1151] Additional financial data (bank account balances, investment portfolios, etc.)

[1152] Specific behavior:

[1153] The server sends a request to the external API and stores the retrieved data back in the database.

[1154] Step 4: Integrate and cleanse the data

[1155] The server integrates all data it has acquired and cleanses missing and outlier values. It uses the Python Pandas library.

[1156] input:

[1157] Financial Data and Additional Financial Data

[1158] output:

[1159] Clean and standardized datasets

[1160] Specific behavior:

[1161] Run the Python script to load the data into a data frame and impute missing values.

[1162] Step 5: Generative AI model generates financial advice

[1163] The server inputs the cleansed data into a generative AI model, such as GPT-3, to generate optimal financial advice.

[1164] input:

[1165] Clean and standardized datasets

[1166] output:

[1167] Generated financial advice (e.g., "Save 1 million yen per year and invest 500,000 yen in a risk-diversified fund")

[1168] Specific behavior:

[1169] Prompt sentences are generated and input into a generative AI model to generate advice.

[1170] Step 6: Submitting and viewing advice

[1171] The server sends the generated financial advice to the user's device, which receives it and displays it on a dashboard. This uses React and Angular, among other tools.

[1172] input:

[1173] Generated Financial Advice

[1174] output:

[1175] Personalized financial advice displayed in a dashboard

[1176] Specific behavior:

[1177] The server sends advice in JSON format, which is displayed on the device's dashboard.

[1178] Step 7: Performing User Actions

[1179] Users can set and take specific actions based on the financial advice they receive, such as setting savings goals or making investments.

[1180] input:

[1181] Actions taken by the user

[1182] output:

[1183] The action taken and its result

[1184] Specific behavior:

[1185] Actions are executed when the user clicks on the dashboard's "Set Savings Goals" or "Invest" buttons. In some cases, Twilio or Stripe APIs are used.

[1186] (Application example 1)

[1187] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1188] In conventional financial management systems, it was difficult for users to input financial data and receive appropriate financial advice based on that data. Furthermore, the lack of means for users to set individual savings goals or run investment simulations made it difficult to provide optimal financial management for each individual user. The present invention aims to solve these problems and provide a system that allows users to easily and effectively manage their finances.

[1189] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1190] In this invention, the server includes means for inputting financial data from a user, means for transmitting the financial data to the server, means for using a generative AI model to analyze the financial data, means for generating customized financial advice using the generative AI model, means for transmitting the customized financial advice to a user terminal, means for displaying the customized financial advice on the user terminal, and means for setting savings goals and executing investment simulations on the user terminal, thereby enabling users to intuitively operate the system, receive customized financial advice, and effectively manage their savings and investments.

[1191] "User" means an individual or legal entity that uses the system, inputs financial data, and receives advice.

[1192] "Financial Data" refers to financial information such as a user's income, expenses, asset status, and risk tolerance.

[1193] "Server" means a computer system that receives financial data submitted by users, utilizes external APIs to obtain additional financial data, analyzes the data, and generates advice.

[1194] A "generative AI model" is a machine learning algorithm that analyzes a user's financial data and generates customized financial advice.

[1195] "Customized financial advice" refers to personalized investment strategies and savings plans created by generative AI models based on a user's financial data.

[1196] "User Device" means the device (e.g., smartphone, tablet, PC, etc.) used by a User to input financial data and view and interact with customized financial advice.

[1197] "External API" means an application program interface used by the Server to obtain additional financial data from external financial institutions or market data providers.

[1198] A "savings goal" refers to a specific savings amount and savings period set by a user, and is a numerical target that the user should aim for financially.

[1199] "Investment simulation" is a process of estimating future returns and risks in advance based on the amount of investment expected by the user.

[1200] "Data analysis" refers to the process of cleaning up financial and external data collected by the server, standardizing it, and inputting it into a generative AI model.

[1201] This invention is a system that allows users to input financial data and uses a generative AI model based on that data to provide customized financial advice. Below, we will explain in detail the processing for each server, terminal, and user.

[1202] Server Processing

[1203] The server receives financial data (income, expenses, asset status, risk tolerance, etc.) sent by the user. This data is received in real time and stored in an internal database. The server then uses external APIs to obtain additional financial data about the user's bank account information and investment portfolio. All of the obtained data is consolidated and cleansed of missing values ​​and outliers. Once the data is clean and standardized, it is input into a generative AI model. The generative AI model has learned from past market trends and data from other users and generates optimal financial advice for the user based on this. The generated advice is formatted in text and graphs and sent back to the user's device.

[1204] Terminal handling

[1205] The terminal provides an interface for user input. The user uses a form to enter their income, expenses, and asset information, which is then sent via the terminal to the server. The entered data sends requests and establishes communication with the server as needed. As financial advice is sent from the server, the terminal receives it in real time and updates its internal data model. It not only displays the advice in a user-friendly format (dashboards and reports), but also provides tools for the user to take specific actions (e.g., setting savings goals or simulating investments).

[1206] User operations

[1207] Users first enter their financial data into the device, including income, expenses, asset status, and risk tolerance, and then send this data to the server. After receiving customized financial advice from the server, users review it and decide on specific actions to apply. For example, they can set a monthly savings amount based on the proposed savings plan, or purchase specific investment products according to the recommended investment strategy. Users can easily manage these actions on the device and track their progress in real time.

[1208] Specific examples

[1209] For example, User A has an annual income of 6 million yen, monthly living expenses of 200,000 yen, and a moderate risk tolerance. User A enters this data into his device and sends it to the server. The server receives the data and uses a generative AI model to generate an optimal savings and investment strategy for User A. Specific advice is generated, such as "save 1 million yen per year and invest 500,000 yen in a risk-diversified fund." The server then sends this advice to User A's device. User A checks the advice on his device, sets actual savings goals through the app, and makes investments. This allows User A to efficiently manage his finances.

[1210] Hardware and software used

[1211] Hardware: Smartphones, tablets, computers

[1212] Software: Python, external APIs (e.g., APIs for retrieving financial data), generative AI models (e.g., GPT-4)

[1213] Prompt Sentence Examples

[1214] "User income: 6 million yen, monthly living expenses: 200,000 yen, assets: 1 million yen, risk tolerance: medium. Based on these criteria, please suggest the optimal savings plan and investment strategy for the user."

[1215] In this manner, the present invention is a system that enables users to receive and implement effective, customized financial advice.

[1216] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1217] Step 1:

[1218] The user enters their financial data into the terminal, including income, expenses, assets, risk tolerance, etc. The data from the user input form is then ready to be sent to the server.

[1219] Input: User's income, expenses, financial situation, risk tolerance

[1220] Output: Financial data entered into the terminal

[1221] Step 2:

[1222] The terminal sends the user's input data to the server, which receives the data in real time and stores it in an internal database.

[1223] Input: Financial data entered into the terminal

[1224] Output: Financial data stored on the server

[1225] Step 3:

[1226] The server uses external APIs to retrieve additional financial data about the user, such as bank account details and investment portfolio, which is then integrated with the existing user data.

[1227] Input: Financial data stored on the server, plus additional financial data retrieved from external APIs

[1228] Output: Consolidated comprehensive user financial data

[1229] Step 4:

[1230] The server cleanses the consolidated financial data and corrects missing and outlier values, ensuring clean and standardized input data for generative AI models.

[1231] Input: Integrated comprehensive user financial data

[1232] Output: Clean, cleaned financial data

[1233] Step 5:

[1234] The server then inputs the cleansed data into a generative AI model, which has learned about past market trends and data from other users, and generates optimal financial advice for the user based on this data.

[1235] Input: Clean, cleaned financial data

[1236] Output: Customized financial advice from a generative AI model

[1237] Step 6:

[1238] The server then formats the generated financial advice in text and graph format and sends it back to the user's terminal, which receives it in real time and displays it in a format that is easy for the user to understand.

[1239] Input: Customized financial advice from a generative AI model

[1240] Output: Financial advice displayed on the terminal

[1241] Step 7:

[1242] The terminal provides tools for users to set savings goals and run investment simulations. Users can set specific savings goals and investment simulations and execute plans based on them.

[1243] Input: Financial advice displayed on user terminal

[1244] Output: Savings goals and investment simulation plans set by the user

[1245] Step 8:

[1246] Users can access financial advice and take concrete actions on savings and investments through their devices, and the app tracks progress in real time, enabling them to effectively manage their finances.

[1247] Input: Savings goals and investment simulation plans set by the user

[1248] Output: Administrative data on savings and investment actions taken

[1249] Through these steps, the present invention enables users to receive and implement effective, customized financial advice.

[1250] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1251] This invention is a system that allows users to input financial data and emotional data, and then provides customized financial advice based on that data using a generative AI model and an emotional engine. The following describes the processing for each server, terminal, and user in detail.

[1252] Server Processing

[1253] 1. Data reception:

[1254] The server first receives financial data (income, expenditure, asset status, risk tolerance, etc.) and emotional data (text messages, voice data, facial expression data, etc.) sent from the user's device. This data is sent to the server in real time and stored in an internal database.

[1255] 2. Data Integration:

[1256] The server uses external APIs to retrieve additional financial data about the user's bank account information and investment portfolio, and then integrates it with the received financial data. After data integration, it performs a cleansing process to correct missing values ​​and outliers.

[1257] 3. Emotion analysis:

[1258] The server uses an emotion engine to analyze the received emotion data, which identifies the user's emotional state (e.g., stress, relief, interest, etc.) and stores it in a database.

[1259] 4. Data Normalization and Analysis:

[1260] The cleansed data is standardized and fed into a generative AI model that takes into account market trends and past user data to generate optimal savings plans and investment strategies for users based on financial and sentiment analysis data.

[1261] 5. Advice generation and transmission:

[1262] The generated advice is formatted as text and graphs and sent to the user's device in an appropriate communication format (e.g., encouraging words or detailed explanations) based on the results of sentiment analysis.

[1263] Terminal handling

[1264] 1. Data Entry:

[1265] The terminal provides a form for users to input financial data and emotional data, including income, expenses, asset information, and risk tolerance, and emotional data including text messages, voice data, and facial expression data.

[1266] 2. Real-time validation and submission:

[1267] Once financial and emotional data is entered, the terminal validates it in real time, prompting the user to re-enter any incomplete data, and then sends the validated data to the server.

[1268] 3. Advice Receiving and Display:

[1269] The terminal receives customized financial advice sent from the server and converts it into an internal data model. The terminal visualizes the received advice and displays it in the form of a dashboard or report. Based on the results of sentiment analysis, the terminal provides advice to the user in an appropriate tone and format.

[1270] 4. Take action:

[1271] The device provides an interface for users to take specific actions based on the advice (such as setting savings goals or making investments), and the progress of the actions taken by the user is updated in real time on the device.

[1272] User operations

[1273] 1. Data Entry:

[1274] Users access the terminal and input their financial data (income, expenses, asset status, risk tolerance) and emotional data (text, voice, facial expressions).

[1275] 2. Data transmission:

[1276] Once the input is complete, the user presses the submit button to send the data to the server, and a notification message is displayed when the data is sent.

[1277] 3. Advice confirmation:

[1278] The user checks the customized financial advice sent from the server along with additional information based on sentiment analysis. For example, if the advice is to "save 1 million yen a year and invest 500,000 yen in a risk-diversified fund," an encouraging message and explanation of the risks will be added based on the results of sentiment analysis.

[1279] 4. Action execution:

[1280] Users can then take specific financial actions based on the advice, such as setting up regular savings or purchasing investment products, via their device.

[1281] 5. Feedback and Replanning:

[1282] The user frequently checks the progress of the action on the device, replans if necessary, and receives feedback based on their emotional state to help generate advice for the next action.

[1283] Specific examples

[1284] For example, User B has an annual income of 8 million yen, monthly living expenses of 250,000 yen, and a high risk tolerance. User B enters this financial data and emotional data, such as "I've been feeling stressed lately," into his / her device and sends it to the server. The server receives this data and analyzes it using a generative AI model and an emotion engine. As a result, it generates advice that is optimal for User B, such as "invest 1.5 million yen per year in a risk-diversified fund and put 500,000 yen into short-term savings." Furthermore, based on the results of the emotion analysis, it also provides "suggestions for low-risk investment strategies to reduce stress" and "detailed explanations to give a sense of security about investing."

[1285] In this way, the present invention provides customized financial advice that takes into account a user's overall financial situation and emotional state, helping users manage their finances more effectively.

[1286] The processing flow will be explained below.

[1287] Server processing flow

[1288] Step 1:

[1289] The server receives financial data and emotional data in real time from the user terminal. The financial data includes income, expenditure, asset status, and risk tolerance, and the emotional data includes text messages, voice data, and facial expression data.

[1290] Step 2:

[1291] The server uses external APIs to obtain additional financial data about the user's bank account details and investment portfolio, which is then stored in an internal database.

[1292] Step 3:

[1293] The server integrates the received financial data with the additional financial data it acquires, and performs a cleansing process to correct missing or outlier values.

[1294] Step 4:

[1295] The server uses an emotion engine to analyze the received emotion data, extracting emotion tags (e.g., stress, relief, interest, etc.) from text messages and voice data.

[1296] Step 5:

[1297] The server standardizes the cleansed financial data and feeds it, along with sentiment analysis results, into the generative AI model. The standardization process includes scaling numerical data and encoding categorical data.

[1298] Step 6:

[1299] The generative AI model analyzes the input data and generates optimal savings plans and investment strategies for users, and the analysis results are converted into text and graph formats.

[1300] Step 7:

[1301] The server then adds a message with an appropriate tone based on the results of sentiment analysis to the generated advice. For example, if a user is under stress, the server might recommend "low-risk investments" and add words of reassurance.

[1302] Step 8:

[1303] The server then sends the prepared advice to the user's device, securely using the HTTPS protocol.

[1304] Terminal processing flow

[1305] Step 1:

[1306] The terminal displays a form for users to enter financial and emotional data, including income, expenses, asset information, risk tolerance, and emotional data (text messages, voice, and facial expressions).

[1307] Step 2:

[1308] As users enter data into forms, the terminal validates the input data in real time and prompts the user to re-enter the data if it is incomplete.

[1309] Step 3:

[1310] The validated data is sent to the server securely using the HTTPS protocol.

[1311] Step 4:

[1312] The terminal receives customized financial advice sent from the server and converts the received data into an internal data model.

[1313] Step 5:

[1314] The device visualizes and displays advice to the user in the form of a dashboard or report, and delivers messages in an appropriate tone based on the results of sentiment analysis.

[1315] Step 6:

[1316] Providing an interface for users to take specific actions based on advice, including interfaces to help set savings goals and make investments.

[1317] Step 7:

[1318] The progress of actions performed by the user is updated in real time and displayed on the device.

[1319] User Process Flow

[1320] Step 1:

[1321] Users access the terminal and input their financial data (income, expenses, asset status, risk tolerance) and emotional data (text messages, voice, facial expressions).

[1322] Step 2:

[1323] Once the input is complete, the user presses the submit button to send the data to the server, and a notification message is displayed when the data has been sent.

[1324] Step 3:

[1325] The user checks the customized financial advice sent from the server and messages based on sentiment analysis. For example, the advice might be "save 1 million yen a year and invest 500,000 yen in a risk-diversified fund," along with suggestions for low-risk investment strategies to reduce stress and encouraging messages.

[1326] Step 4:

[1327] Users take specific financial actions based on the advice, such as setting up regular savings or purchasing investment products.

[1328] Step 5:

[1329] Users frequently check the progress of their actions on their devices and make replanning and adjustments as needed.

[1330] Step 6:

[1331] Users can provide feedback on their achieved goals and ongoing plans via their devices, which will be used to generate the next advice. This feedback will be used to provide even more accurate advice.

[1332] Example 2

[1333] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1334] In modern financial management, not only individual users' financial data but also their emotional state at any given time is an important factor. However, conventional financial advisory systems have been unable to fully utilize emotional data, making it difficult to provide customized advice. This has led to issues such as users feeling stressed when making appropriate financial management and investment decisions, making it difficult to manage their finances efficiently.

[1335] The identification process by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means. In this invention, the server includes: means for inputting financial data and emotion data from a user; means for transmitting the financial data and emotion data to the server; means for using a generative AI model and an emotion engine to analyze the data; means for generating customized financial advice using the generative AI model and emotion engine; means for transmitting the customized financial advice to a user terminal; means for displaying the customized financial advice on the user terminal; means for the server to obtain additional financial data using an external API and integrate the data; means for the server to analyze the emotion data and identify the user's emotional state; means for the user terminal to validate the financial data and emotion data in real time and notify the user of any input errors; means for the customized financial advice to be provided to the user in a form that reflects the emotion analysis results; means for the user terminal to support the user in taking action based on the advice; and means for the user to provide feedback that is used to generate the next advice. This enables more personalized financial management for users and provides optimal advice that takes into account their emotional state and market trends.

[1336] markdown

[1337] A "user" is an individual or entity that uses the system to input financial and emotional data and receive customized financial advice.

[1338] "Financial data" refers to data relating to the user's economic situation, such as income, expenses, asset status, and risk tolerance.

[1339] "Emotion data" refers to data such as text messages, voice data, and facial expression data that indicate the user's emotional state.

[1340] A "server" is a central computing unit that receives data sent from user terminals, performs analysis, and generates financial advice.

[1341] A "generative AI model" is a machine learning model that takes into account market trends and historical data to generate customized financial advice based on input data.

[1342] An "emotion engine" is a software module for analyzing emotion data and identifying the user's emotional state.

[1343] A "user terminal" is an electronic device through which a user inputs financial and emotional data and receives customized financial advice transmitted from a server.

[1344] An "external API" is an interface for obtaining data from other systems or services.

[1345] "Real-time validation" is a process that immediately checks input data and prompts correction of any incomplete data.

[1346] "Customized financial advice" is advice generated by a generative AI model and emotion engine based on a user's individual financial situation and emotional state.

[1347] A "means for supporting behavioral execution" is a part of the system that provides an interface or functionality for users to take specific actions based on customized financial advice.

[1348] "Feedback" refers to opinions and evaluations provided by users to the system, and is information that will be used to generate the next piece of advice.

[1349] MODE FOR CARRYING OUT THE INVENTION

[1350] The present invention is a system for providing optimal financial advice to users based on financial data and emotional data. The processing performed by the server, terminal, and user will be described in detail below.

[1351] Server Processing

[1352] Data reception

[1353] The server first receives financial data (income, expenses, asset status, risk tolerance, etc.) and emotional data (text messages, voice data, facial expression data, etc.) sent from the user's device. This data is sent to the server in real time and stored in an internal database. For example, if a user sends data from their device such as "monthly income of 500,000 yen, monthly expenses of 300,000 yen," this information is stored in the receiving buffer.

[1354] Data Integration

[1355] The server uses external APIs to retrieve additional financial data about the user's bank account information and investment portfolio, and then integrates it with the original financial data. This process also includes a cleansing process to automatically correct missing data and outliers. For example, if a user has multiple accounts and their account information is retrieved via an API, the collected data can be integrated with the input data.

[1356] Emotion analysis

[1357] The server uses an emotion engine to analyze the received emotion data. Through this analysis, the user's emotional state (e.g., stress, relief, interest, etc.) is identified and stored in a database. For example, from a user's message such as "I'm tired today," the emotion engine determines the user's stress level.

[1358] Data normalization and analysis

[1359] The cleansed data is then standardized and fed into a generative AI model, which takes into account market trends and historical data to generate optimal savings plans and investment strategies for users based on financial and sentiment data. For example, historical market data could be used to suggest optimal investment options for users.

[1360] Advice generation and delivery

[1361] The generated advice is then formatted as text and graphs and sent to the user in an appropriate communication format based on the results of the emotion analysis. For example, advice containing encouraging words and detailed explanations may be sent.

[1362] Terminal handling

[1363] Data Entry

[1364] The device provides a form for users to input financial and emotional data. Financial data includes income, expenses, asset information, and risk tolerance, while emotional data includes text messages, voice data, and facial expression data. For example, the device has text boxes for inputting "income" and "expenses," as well as voice recording and facial expression recognition functions for expressing "emotions."

[1365] Real-time validation and submission

[1366] When financial and emotional data is entered, the terminal validates it in real time and displays a warning if the data is incomplete. For example, if a negative income value is entered, an error message is displayed prompting the user to correct it. The terminal also sends the validated data to the server.

[1367] Advice reception and display

[1368] The device receives customized financial advice sent from the server and displays it visually, including graphical dashboards and detailed text messages. For example, if the advice is to "save 1 million yen a year and invest 500,000 yen in a risk-diversified fund," it will visualize it in pie charts and bar graphs.

[1369] Action Execution

[1370] The device provides an interface for users to take action based on the advice. For example, a "Set Savings Goal" button or an "Invest" button are displayed, and users can tap the button to start taking specific action.

[1371] User operations

[1372] Data Entry

[1373] The user uses the device to input their own financial data (income, expenditure, asset status, risk tolerance) and emotional data (text, voice, facial expression). For example, they input "income 500,000 yen, expenditure 300,000 yen" and also emotional data such as "I've been under a lot of stress lately."

[1374] Data transmission

[1375] The data you have entered will be sent to the server. When you press the send button, the progress will be displayed and you will be notified when the data has been sent.

[1376] Advice confirmation

[1377] Check the customized advice sent from the server. For example, "Invest 1.5 million yen a year in a risk-diversified fund and put 500,000 yen into short-term savings," along with an encouraging message based on the results of sentiment analysis.

[1378] Action execution

[1379] The user then takes specific actions based on the advice, such as purchasing an investment fund, and checks the progress of the action on the device.

[1380] Feedback and replanning

[1381] Users can track the progress of their actions, reorganize their plans if necessary, and provide feedback on their emotional state to help inform the next recommendation. For example, they can provide feedback like, "I'm happy with my current investments, but I'd like to see a less risky option," which will be reflected in the next recommendation.

[1382] Specific examples

[1383] For example, User B has an annual income of 8 million yen, monthly living expenses of 250,000 yen, and a high risk tolerance. User B enters this financial data and emotional data, such as "I've been feeling stressed lately," into his / her device and sends it to the server. The server analyzes the received data and generates advice such as "invest 1.5 million yen per year in a risk-diversified fund and put the remaining 500,000 yen into short-term savings." Furthermore, based on the results of the emotional analysis, the advice is accompanied by "low-risk investment strategies to reduce stress" and "detailed explanations to give a sense of security about investing." This advice is displayed on the device, and the user then takes specific action (purchase an investment fund) through the device.

[1384] Prompt Sentence Examples

[1385] "Enter information about your income and expenses, financial situation, and risk tolerance, and provide your recent emotional state via text, voice, and facial expressions."

[1386] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1387] System program processing flow

[1388] Server Processing

[1389] Step 1: Receiving data

[1390] The server receives financial data (income, expenditure, asset status, risk tolerance) and emotional data (text messages, voice data, facial expression data) sent from the user's device. This data is stored in a receiving buffer in real time. For example, information entered by a user as income of 500,000 yen and expenditure of 300,000 yen is saved.

[1391] Input: Financial and emotional data from user devices

[1392] Output: Data stored in the receive buffer

[1393]

[1394] Step 2: Data Integration

[1395] The server uses external APIs to retrieve additional financial data about the user's bank account information and investment portfolio, and combines it with the incoming financial data. This combined data undergoes cleansing processes to correct missing and outlier values. For example, data retrieved from financial institution APIs can be combined with user input data.

[1396] Input: Incoming financial data and financial data obtained from external APIs

[1397] Output: Unified and cleansed data

[1398]

[1399] Step 3: Sentiment Analysis

[1400] The server uses an emotion engine to analyze the received emotion data. This analysis identifies the user's emotional state (e.g., stress, relief, interest) and stores it in a database. For example, if a user says, "I'm tired today," the emotion engine analyzes the user's stress level.

[1401] Input: Received emotion data

[1402] Output: Parsed emotional state data

[1403]

[1404] Step 4: Data standardization and analysis

[1405] The server standardizes the cleansed data and inputs it into a generative AI model, which takes into account market trends and historical data, and generates optimal savings plans and investment strategies for users based on financial and sentiment data. For example, it might suggest optimal investments based on user data combined with market trend data.

[1406] Input: Cleansed and standardized data

[1407] Output: Analysis results from the generative AI model (savings plan and investment strategy)

[1408]

[1409] Step 5: Advice generation and sending

[1410] The generated advice is then formatted as text and graphs and sent to the user in an appropriate communication format based on the results of the emotion analysis. For example, advice containing encouraging words and detailed explanations may be sent.

[1411] Input: Analysis results and emotional state data

[1412] Output: Customized financial advice sent to the user

[1413] Terminal handling

[1414] Step 1: Data entry

[1415] The device provides a form for users to enter financial and emotional data, including text boxes for entering income and expenses, as well as voice recording and facial recognition capabilities.

[1416] Input: User financial and emotional data input

[1417] Output: Data entered in the input form

[1418]

[1419] Step 2: Real-time validation and submission

[1420] The terminal validates the data entered in real time and displays a warning if the data is incomplete. For example, if a negative income value is entered, an error message is displayed and the user is prompted to correct the data. After validation is complete, the data is sent to the server.

[1421] Input: User-entered financial and sentiment data

[1422] Output: Send validated data to the server

[1423]

[1424] Step 3: Receive and view advice

[1425] The terminal receives customized financial advice sent from the server and displays it in a visualized format (dashboard and graphs). For example, advice such as "save 1 million yen a year and invest 500,000 yen in a risk-diversified fund" is visualized.

[1426] Input: Customized financial advice from the server

[1427] Output: Visualized advice display

[1428]

[1429] Step 4: Take Action

[1430] The device provides an interface for users to take specific actions based on the advice, such as a "Set Savings Goal" button or an "Invest" button that users can tap to start the action.

[1431] Enter: customized financial advice

[1432] Output: User action execution

[1433] User operations

[1434] Step 1: Data entry

[1435] The user inputs financial data (income, expenditure, asset status, risk tolerance) and emotional data (text, voice, facial expression) into the terminal. For example, the user inputs "income 500,000 yen, expenditure 300,000 yen" and emotional data such as "I've been under a lot of stress lately."

[1436] Input: Financial and sentiment data

[1437] Output: Filling in the input form

[1438]

[1439] Step 2: Send data

[1440] The data you have entered will be sent to the server. When you press the send button, the progress will be displayed and you will be notified when the data has been sent.

[1441] Input: Data entered into an input form

[1442] Output: Send data to the server

[1443]

[1444] Step 3: Check the advice

[1445] The user then checks the customized financial advice sent from the server, such as "Invest 1.5 million yen a year in a risk-diversified fund and put 500,000 yen into short-term savings," along with an encouraging message based on the results of sentiment analysis.

[1446] Input: Customized financial advice sent from the server

[1447] Output: Check the visualized advice

[1448]

[1449] Step 4: Take action

[1450] The user then takes specific actions based on the advice, such as purchasing an investment fund, and checks the progress of the action on the device.

[1451] Enter: customized financial advice

[1452] Output: Implementing specific actions

[1453]

[1454] Step 5: Feedback and replanning

[1455] Users can track their progress and adjust their plans as needed. The system also provides feedback based on their emotional state, such as, "I'm happy with my current investments, but I'd like to see a less risky option."

[1456] Input: Action execution results and emotional feedback

[1457] Output: Data used to generate next advice

[1458]

[1459] (Application example 2)

[1460] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1461] In conventional factory work, workers' emotional states have a significant impact on work performance and safety measures, but there is a lack of effective ways to manage them in real time. Furthermore, there is a lack of technology to provide optimal advice to each individual worker. Therefore, there is a need to improve work performance and strengthen safety measures.

[1462] The specific processing by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for inputting financial data and emotional data from a user, means for transmitting the financial data and emotional data to the server, means for using a generative AI model and an emotion engine that analyzes the financial data and emotional data, means for generating customized financial advice and work performance improvement advice using the generative AI model and emotion engine, means for transmitting the customized financial advice and work performance improvement advice to a user terminal, and means for displaying the customized financial advice and work performance improvement advice on the user terminal. This makes it possible to analyze the emotional state of a worker in real time and provide optimal advice.

[1463] "User" refers to a person or organization that uses the system.

[1464] "Financial Data" refers to financial information about an individual or organization, such as income, expenses, assets, and risk tolerance.

[1465] "Emotional data" refers to information that represents an emotional state, such as text messages, voice data, and facial expression data.

[1466] "Server" refers to the central processing unit that receives and analyzes data sent by the User and retransmits the generated information to the User.

[1467] "Generative AI model" refers to a machine learning model that analyzes a user's financial and emotional data to generate customized advice.

[1468] "Emotion engine" refers to an analysis system for analyzing emotion data and identifying a user's emotional state.

[1469] "User terminal" refers to a device used by a user to input data or receive and display information sent from the server.

[1470] "External Data Acquisition Means" refers to means for acquiring and integrating additional information from external data sources.

[1471] "Work data" refers to data related to the progress and performance of work in a factory or on-site.

[1472] "Operation data" refers to data relating to the operating status of machinery and equipment within a factory.

[1473] The present invention is a system that utilizes a user's financial data and emotion data to provide customized financial advice, including work performance improvement and safety measures, using a generative AI model and emotion engine. Specific processing for each server, terminal, and user is described below.

[1474] Server Processing

[1475] 1. Data Reception

[1476] The server receives financial data (e.g., income, expenses, asset status, risk tolerance) and emotional data (e.g., text messages, voice data, facial expression data) sent from the user's device. This data is sent to the server in real time and stored in an internal database.

[1477] 2. External Data Acquisition and Data Integration

[1478] The server uses external data acquisition means to acquire work data and equipment operation data. The acquired data is integrated and cleansed. This process uses data integration programs and data cleaning algorithms in a Python environment.

[1479] 3. Sentiment Analysis and Data Analysis

[1480] The server analyzes the emotion data using an emotion engine (e.g., Spacy, IBM Watson). The analysis results are combined with financial data and input into a generative AI model (e.g., TensorFlow, PyTorch). The generative AI model uses this data to generate customized financial advice and work performance improvement advice.

[1481] 4. Advice Generation and Delivery

[1482] The generated advice is formatted as text and graphs and sent to the user's device in an appropriate communication format (e.g., detailed explanations that provide a sense of security) based on the results of sentiment analysis.

[1483] Terminal handling

[1484] 1. Data Entry

[1485] The device provides an interface for users to input financial and emotional data, including text fields, voice input, and facial expression input using a camera.

[1486] 2. Data Transmission

[1487] The entered data is sent from the terminal to the server. This process includes a data transmission program and validation check function.

[1488] 3. Receiving and displaying advice

[1489] The customized advice sent from the server is received by the device, visualized, and displayed to the user. This user interface includes dashboards and reports. In particular, the advice for improving work performance is displayed on smart glasses.

[1490] User operations

[1491] 1. Data Entry

[1492] The user accesses the input interface of the terminal and inputs their own financial data (e.g., income, expenses, asset status, risk tolerance) and emotional data (e.g., text, voice, facial expression).

[1493] 2. Data Transmission

[1494] Once the input is complete, the user presses the submit button to send the data to the server, and a notification message is displayed when the data is sent.

[1495] 3. Check and implement the advice

[1496] After checking the customized advice sent from the server, the user can take specific actions based on the advice, for example, a worker can adjust their work performance or take a break according to the advice displayed on the smart glasses.

[1497] Specific examples

[1498] For example, if Worker A working in a factory comments that he has been under a lot of stress lately, and this emotional state is also confirmed by facial expression data, the server will generate advice to Worker A based on this, recommending that he prioritize low-risk work and take a temporary break. This advice is displayed on the smart glasses.

[1499] Prompt Sentence Examples

[1500] "Design a real-time emotion analysis application for factory workers. This application requires workers to input their emotion data (e.g., stress or relief) and work performance data, and then displays appropriate advice in real time on smart glasses. Data analysis is based on a generative AI model and an emotion engine. Please also generate the corresponding Python code."

[1501] The present invention is a system that handles emotional data and financial data in an integrated manner to provide customized advice to factory workers, thereby improving productivity and strengthening safety measures.

[1502] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1503] Step 1: The user inputs financial and emotional data into the terminal.

[1504] Users use the device's interface to input financial data such as income, expenses, asset status, and risk tolerance. They also input text messages, voice data, and facial expression data that indicate their emotional state via a camera or microphone. The input data is temporarily stored in the device.

[1505] Step 2: The terminal performs real-time validation and sends the data to the server.

[1506] The terminal validates the entered financial and emotional data in real time to check for incomplete data or outliers. Once validated, the data is encrypted and sent to the server, where it checks the input data (income, expenses, emotional state) and either displays an error message or converts it into a format that can be sent.

[1507] Step 3: The server receives the data and stores it in the database.

[1508] The server receives the financial and emotional data sent from the device and stores it in an internal database, which is used in subsequent processing steps.

[1509] Step 4: The server acquires the additional data using an external data acquisition means.

[1510] The server makes API calls to retrieve work data and equipment operation data from external sources. The retrieved data is integrated with internal data and stored in a database. Additional data (such as equipment operation status) is retrieved and integrated with the existing database.

[1511] Step 5: The server performs the cleansing process.

[1512] The server performs a cleansing process on all the acquired data, filling in missing values ​​and correcting outliers, using a Python data analysis library.

[1513] Step 6: The server performs sentiment analysis.

[1514] The server analyzes the emotional data using an emotion engine (e.g., Spacy, IBM Watson). The analysis results are stored in a database as the user's emotional state (stress, relief, etc.). Input: Emotional data (text, voice, facial expression), Output: Emotional state (stress, elation, etc.).

[1515] Step 7: The server inputs the data into the generative AI model and performs analysis.

[1516] The cleansed data and sentiment analysis results are input into a generative AI model. The generative AI model (e.g., TensorFlow, PyTorch) analyzes the financial data and integrated data to generate customized financial and work performance improvement advice. Input: Financial data, sentiment data, work data. Output: Customized advice (savings plans, investment strategies, work instructions, etc.).

[1517] Step 8: The server sends the generated advice.

[1518] The generated advice is formatted as text and graphs and sent to the user terminal in an appropriate communication format. Input: Advice data (text, graphs), Output: Transmission to the user terminal.

[1519] Step 9: The device receives and displays the advice.

[1520] The user device receives the customized advice sent from the server and visualizes and displays it. The display format is a dashboard or report. In particular, advice for improving work performance is displayed on smart glasses, etc. Input: Received data (advice), Output: Visual display (graph, text).

[1521] Step 10: The user acts on the advice.

[1522] The user checks the customized advice displayed on the device and takes specific actions based on it, such as creating a savings plan or changing the priorities of tasks. Input: Check the advice, Output: Specific actions (savings, investment, task change, etc.).

[1523] The specific processing unit 290 transmits the result of the specific processing to the headset type terminal 314. In the headset type terminal 314, the control unit 46A causes the speaker 240 and the display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1524] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1525] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the headset type terminal 314.

[1526] [Fourth embodiment]

[1527] FIG. 7 shows an example of the configuration of a data processing system 410 according to the fourth embodiment.

[1528] 7, a data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[1529] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1530] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a control target 443. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the control target 443 are also connected to the bus 52.

[1531] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1532] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1533] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1534] The control object 443 includes a display device, LEDs in the eyes, and motors for driving the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the emotions of the robot 414 can be expressed by controlling these motors. In addition, the facial expressions of the robot 414 can also be expressed by controlling the light emission state of the LEDs in the eyes of the robot 414.

[1535] Fig. 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Fig. 8, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1536] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1537] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1538] In the robot 414, the processor 46 performs the reception output process. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1539] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1540] The present invention is a system that allows users to input financial data and uses a generative AI model based on that data to provide customized financial advice. The following describes the processing for each server, terminal, and user in detail.

[1541] Server Processing

[1542] The server first receives financial data (income, expenses, assets, risk tolerance, etc.) submitted by the user. This data is received in real time and stored in an internal database. The server then utilizes external APIs to obtain additional financial data about the user's bank account information and investment portfolio.

[1543] All acquired data is consolidated and cleansed of missing or outlier values. Once the data is clean and standardized, it is fed into a generative AI model. The generative AI model has learned from past market trends and data from other users, and uses this information to generate optimal financial advice for the user. The generated advice is then formatted in text and graphs and sent back to the user's device.

[1544] Terminal handling

[1545] The terminal provides an interface for user input. The user uses a form to enter their income, expenses, and asset information, which is then sent to the server via the terminal. The entered data sends requests as needed to establish communication with the server.

[1546] As financial advice is sent from the server, the device receives it in real time, updates its internal data model, and presents it in a user-friendly format (dashboards and reports). It also provides tools for users to take specific actions, such as setting savings goals or making investments.

[1547] User operations

[1548] Users first input their financial data into the device, including income, expenses, asset status, and risk tolerance, and then send this data to the server. Once the server sends them customized financial advice, the user can review it and decide on specific actions to apply.

[1549] For example, users can set monthly savings amounts based on a proposed savings plan, or purchase specific investment products according to a recommended investment strategy. Users can easily manage these actions on their device and track their progress in real time.

[1550] Specific examples

[1551] For example, User A has an annual income of 6 million yen, monthly living expenses of 200,000 yen, and a moderate risk tolerance. User A enters this data into a terminal and sends it to the server. The server receives the data and uses a generative AI model to generate an optimal savings and investment strategy for User A. Specific advice such as "save 1 million yen per year and invest 500,000 yen in a risk-diversified fund" is generated.

[1552] The server then sends this advice to User A's device. User A checks the advice on the device, sets savings goals through the app, and makes investments. This allows User A to efficiently manage their finances.

[1553] In this manner, the present invention is a system that enables users to receive and implement effective, customized financial advice.

[1554] The processing flow will be explained below.

[1555] Server processing flow

[1556] Step 1:

[1557] The server receives financial data sent from the user terminal, including income, expenses, asset status, risk tolerance, etc.

[1558] Step 2:

[1559] The server uses external APIs to obtain additional financial data about the user, such as bank account details and investment portfolios, and receives the response of the external API and stores it in an internal database.

[1560] Step 3:

[1561] The server integrates the received financial data with the additional financial data it has acquired. After the data integration, it performs a cleansing process to correct missing and outlier values.

[1562] Step 4:

[1563] The server standardizes the cleansed data and inputs it into the generative AI model. The standardization process includes scaling numerical data and encoding categorical data.

[1564] Step 5:

[1565] A generative AI model analyzes input data and generates optimal savings and investment strategies for users, taking into account historical market trends and data from other users.

[1566] Step 6:

[1567] The server converts the generated advice into text and graphical formats, making it easily understandable to the user.

[1568] Step 7:

[1569] The server then sends the prepared advice to the user's device, securely using the HTTPS protocol.

[1570] Terminal processing flow

[1571] Step 1:

[1572] The terminal displays a form for the user to enter financial data, including income, expenses, asset information, and risk tolerance.

[1573] Step 2:

[1574] As users enter data into forms, the terminal validates the input data in real time and prompts the user to re-enter any incomplete data.

[1575] Step 3:

[1576] The terminal sends the validated data to the server using the HTTPS protocol to ensure the safety of the user data.

[1577] Step 4:

[1578] When financial advice is sent from the server, the terminal receives it and converts the received data into an internal data model.

[1579] Step 5:

[1580] The device visualizes the advice it receives and displays it in the form of a dashboard or report that is easy for the user to understand.

[1581] Step 6:

[1582] The terminal provides an interface that allows users to take specific actions based on the advice (such as setting savings goals or making investments).

[1583] User Process Flow

[1584] Step 1:

[1585] The user accesses the terminal and inputs their income, expenses, asset status, and risk tolerance, entering the required information into the input form one by one.

[1586] Step 2:

[1587] Once the input is complete, the user presses the submit button to send the data to the server, and a notification message is displayed when the data is sent.

[1588] Step 3:

[1589] When the advice is sent from the server, the user checks it on the device, understands the advice, and prepares by taking notes of any necessary parts.

[1590] Step 4:

[1591] Users take specific financial actions based on the advice, such as setting up regular savings or purchasing investment products.

[1592] Step 5:

[1593] Users frequently check the progress of their actions on their devices and make replanning and adjustments as needed.

[1594] Step 6:

[1595] Users can provide feedback on their achieved goals and ongoing plans via the device, and ask for further advice. This feedback will help provide more accurate advice.

[1596] Example 1

[1597] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1598] Conventional financial advice systems face the problem of being unable to provide adequately customized advice based on a user's individual financial data. In particular, complex processes are required, such as real-time data integration and cleansing, and the acquisition of additional data using external APIs. However, because these processes are not sufficiently optimized, users are unable to receive highly accurate advice.

[1599] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1600] In this invention, the server includes means for inputting financial data from a user, means for transmitting the financial data to the server, means for the server to obtain additional financial data using an external API, means for integrating the financial data with the additional financial data and cleansing missing values ​​and outliers, means for generating customized financial advice using the generative AI model, means for transmitting the customized financial advice to a user terminal, and means for displaying the customized financial advice on the user terminal, thereby enabling more accurate customized financial advice to be provided to users in real time.

[1601] A "user" is a person or entity that inputs financial data and receives customized financial advice.

[1602] "Financial data" is a general term for information about a user's financial situation, such as income, expenses, asset status, and risk tolerance.

[1603] A "server" is a computer system for receiving and processing financial data submitted by users.

[1604] "External API" means an application program interface for accessing third-party services to obtain additional financial data.

[1605] "Financial data" is a general term for information about bank accounts and investment portfolios.

[1606] "Integration" refers to the process of bringing together data obtained from different data sources.

[1607] "Cleansing" is a data preparation process that removes missing and outliers from data and makes any necessary corrections.

[1608] A "generative AI model" is an algorithm or system that uses artificial intelligence to generate customized financial advice.

[1609] "Customized financial advice" is personalized advice created based on a user's individual financial data.

[1610] A "user terminal" is a device through which a user inputs financial data and receives financial advice sent from the server.

[1611] A "prompt" is a string of text data that is input into a generative AI model, and the AI ​​uses this to determine the output it generates.

[1612] The present invention is a system that allows users to input their own financial data and uses a generative AI model to provide customized financial advice. The following describes the processing for each server, terminal, and user in detail.

[1613] Server Processing

[1614] The server first receives financial data (income, expenses, asset status, risk tolerance, etc.) sent by the user. This data is received in real time and stored in an internal database, for example, using a relational database such as MySQL or PostgreSQL.

[1615] The server then uses external APIs to retrieve additional financial data, such as the user's bank account information and investment portfolio, using common APIs that are widely used for financial information.

[1616] All acquired data is consolidated and processed to remove missing or outlier values. Data cleansing is performed using Python or R libraries (such as Pandas). Once the data is clean and standardized, it is input into a generative AI model. This generative AI model is a natural language generation model, such as GPT-3 or BERT, which has been trained on past market trends and data from other users. Based on this, it generates optimal financial advice for the user. The generated advice is then formatted in text or graph form, visualized using tools such as Matplotlib, and sent back to the user's device.

[1617] Terminal handling

[1618] The terminal provides an interface for user input. The user uses a form to enter their income, expenses, and asset information, which is then sent to the server via the terminal. The entered data is sent to the server in real time using AJAX requests or WebSockets.

[1619] As financial advice is sent from the server, the device receives it in real time and updates its internal data model. It builds a dashboard using front-end frameworks such as React or Angular to display the advice in a format that's easy for users to understand. It also provides tools for users to take specific actions, such as setting savings goals or making investments. This includes interfaces for taking actions directly using APIs such as Twilio and Stripe.

[1620] User operations

[1621] Users first input their financial data into the device, including income, expenses, asset status, and risk tolerance, and then send this data to the server. Once the server sends them customized financial advice, the user can review it and decide on specific actions to apply.

[1622] For example, users can set monthly savings amounts based on a proposed savings plan, or purchase specific investment products according to a recommended investment strategy. Users can easily manage these actions on their device and track their progress in real time. This includes tracking user behavior using analytics tools such as Google Analytics and Mixpanel.

[1623] Specific examples

[1624] For example, User A has an annual income of 6 million yen, monthly living expenses of 200,000 yen, and a moderate risk tolerance. User A enters this data into a device and sends it to the server. The server receives the data and uses a generative AI model to generate an optimal savings and investment strategy for User A. The advice will be specific, such as "save 1 million yen per year and invest 500,000 yen in a risk-diversified fund."

[1625] An example of a prompt sentence to input to the generative AI model is as follows:

[1626] "User A's annual income is 6 million yen, and his monthly living expenses are 200,000 yen. User A has a medium risk tolerance. Based on this, please suggest the optimal savings and investment strategy."

[1627] This prompt is then fed into a generative AI model to generate specific financial advice for the user, helping them to manage their finances more efficiently.

[1628] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1629] Step 1: User enters financial data

[1630] The user enters their income, expenses, asset status, and risk tolerance into a form on the device, using text fields, drop-down lists, check boxes, etc.

[1631] input:

[1632] Income: 6 million yen

[1633] Expenses: 200,000 yen per month

[1634] Asset status: 5 million yen

[1635] Risk tolerance: Medium

[1636] output:

[1637] This financial data is entered into the terminal and converted into JSON format.

[1638] Specific behavior:

[1639] When the user clicks the "Submit" button on the form, the data is packaged in JSON format.

[1640] Step 2: Sending and Receiving Data

[1641] The terminal sends the entered financial data to the server, which receives it and stores it in a database. Real-time transmission is achieved using AJAX requests and WebSockets.

[1642] input:

[1643] Financial data in JSON format: {Annual income: $6,000,000, Monthly living expenses: $2,000, Asset status: $5,000, Risk tolerance: Medium}

[1644] output:

[1645] The server stores the data in a database.

[1646] Specific behavior:

[1647] The terminal sends JSON data, which the server receives and stores in a MySQL database.

[1648] Step 3: Obtaining additional data via an external API

[1649] The server uses an external API to retrieve the user's bank account information and investment portfolio data. It uses a financial information API (e.g., Plaid).

[1650] input:

[1651] User credentials

[1652] output:

[1653] Additional financial data (bank account balances, investment portfolios, etc.)

[1654] Specific behavior:

[1655] The server sends a request to the external API and stores the retrieved data back in the database.

[1656] Step 4: Integrate and cleanse the data

[1657] The server integrates all data it has acquired and cleanses missing and outlier values. It uses the Python Pandas library.

[1658] input:

[1659] Financial Data and Additional Financial Data

[1660] output:

[1661] Clean and standardized datasets

[1662] Specific behavior:

[1663] Run the Python script to load the data into a data frame and impute missing values.

[1664] Step 5: Generative AI model generates financial advice

[1665] The server inputs the cleansed data into a generative AI model, such as GPT-3, to generate optimal financial advice.

[1666] input:

[1667] Clean and standardized datasets

[1668] output:

[1669] Generated financial advice (e.g., "Save 1 million yen per year and invest 500,000 yen in a risk-diversified fund")

[1670] Specific behavior:

[1671] Prompt sentences are generated and input into a generative AI model to generate advice.

[1672] Step 6: Submitting and viewing advice

[1673] The server sends the generated financial advice to the user's device, which receives it and displays it on a dashboard. This uses React and Angular, among other tools.

[1674] input:

[1675] Generated Financial Advice

[1676] output:

[1677] Personalized financial advice displayed in a dashboard

[1678] Specific behavior:

[1679] The server sends advice in JSON format, which is displayed on the device's dashboard.

[1680] Step 7: Performing User Actions

[1681] Users can set and take specific actions based on the financial advice they receive, such as setting savings goals or making investments.

[1682] input:

[1683] Actions taken by the user

[1684] output:

[1685] The action taken and its result

[1686] Specific behavior:

[1687] Actions are executed when the user clicks on the dashboard's "Set Savings Goals" or "Invest" buttons. In some cases, Twilio or Stripe APIs are used.

[1688] (Application example 1)

[1689] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1690] In conventional financial management systems, it was difficult for users to input financial data and receive appropriate financial advice based on that data. Furthermore, the lack of means for users to set individual savings goals or run investment simulations made it difficult to provide optimal financial management for each individual user. The present invention aims to solve these problems and provide a system that allows users to easily and effectively manage their finances.

[1691] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1692] In this invention, the server includes means for inputting financial data from a user, means for transmitting the financial data to the server, means for using a generative AI model to analyze the financial data, means for generating customized financial advice using the generative AI model, means for transmitting the customized financial advice to a user terminal, means for displaying the customized financial advice on the user terminal, and means for setting savings goals and executing investment simulations on the user terminal, thereby enabling users to intuitively operate the system, receive customized financial advice, and effectively manage their savings and investments.

[1693] "User" means an individual or legal entity that uses the system, inputs financial data, and receives advice.

[1694] "Financial Data" refers to financial information such as a user's income, expenses, asset status, and risk tolerance.

[1695] "Server" means a computer system that receives financial data submitted by users, utilizes external APIs to obtain additional financial data, analyzes the data, and generates advice.

[1696] A "generative AI model" is a machine learning algorithm that analyzes a user's financial data and generates customized financial advice.

[1697] "Customized financial advice" refers to personalized investment strategies and savings plans created by generative AI models based on a user's financial data.

[1698] "User Device" means the device (e.g., smartphone, tablet, PC, etc.) used by a User to input financial data and view and interact with customized financial advice.

[1699] "External API" means an application program interface used by the Server to obtain additional financial data from external financial institutions or market data providers.

[1700] A "savings goal" refers to a specific savings amount and savings period set by a user, and is a numerical target that the user should aim for financially.

[1701] "Investment simulation" is a process of estimating future returns and risks in advance based on the amount of investment expected by the user.

[1702] "Data analysis" refers to the process of cleaning up financial and external data collected by the server, standardizing it, and inputting it into a generative AI model.

[1703] This invention is a system that allows users to input financial data and uses a generative AI model based on that data to provide customized financial advice. Below, we will explain in detail the processing for each server, terminal, and user.

[1704] Server Processing

[1705] The server receives financial data (income, expenses, asset status, risk tolerance, etc.) sent by the user. This data is received in real time and stored in an internal database. The server then uses external APIs to obtain additional financial data about the user's bank account information and investment portfolio. All of the obtained data is consolidated and cleansed of missing values ​​and outliers. Once the data is clean and standardized, it is input into a generative AI model. The generative AI model has learned from past market trends and data from other users and generates optimal financial advice for the user based on this. The generated advice is formatted in text and graphs and sent back to the user's device.

[1706] Terminal handling

[1707] The terminal provides an interface for user input. The user uses a form to enter their income, expenses, and asset information, which is then sent via the terminal to the server. The entered data sends requests and establishes communication with the server as needed. As financial advice is sent from the server, the terminal receives it in real time and updates its internal data model. It not only displays the advice in a user-friendly format (dashboards and reports), but also provides tools for the user to take specific actions (e.g., setting savings goals or simulating investments).

[1708] User operations

[1709] Users first enter their financial data into the device, including income, expenses, asset status, and risk tolerance, and then send this data to the server. After receiving customized financial advice from the server, users review it and decide on specific actions to apply. For example, they can set a monthly savings amount based on the proposed savings plan, or purchase specific investment products according to the recommended investment strategy. Users can easily manage these actions on the device and track their progress in real time.

[1710] Specific examples

[1711] For example, User A has an annual income of 6 million yen, monthly living expenses of 200,000 yen, and a moderate risk tolerance. User A enters this data into his device and sends it to the server. The server receives the data and uses a generative AI model to generate an optimal savings and investment strategy for User A. Specific advice is generated, such as "save 1 million yen per year and invest 500,000 yen in a risk-diversified fund." The server then sends this advice to User A's device. User A checks the advice on his device, sets actual savings goals through the app, and makes investments. This allows User A to efficiently manage his finances.

[1712] Hardware and software used

[1713] Hardware: Smartphones, tablets, computers

[1714] Software: Python, external APIs (e.g., APIs for retrieving financial data), generative AI models (e.g., GPT-4)

[1715] Prompt Sentence Examples

[1716] "User income: 6 million yen, monthly living expenses: 200,000 yen, assets: 1 million yen, risk tolerance: medium. Based on these criteria, please suggest the optimal savings plan and investment strategy for the user."

[1717] In this manner, the present invention is a system that enables users to receive and implement effective, customized financial advice.

[1718] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1719] Step 1:

[1720] The user enters their financial data into the terminal, including income, expenses, assets, risk tolerance, etc. The data from the user input form is then ready to be sent to the server.

[1721] Input: User's income, expenses, financial situation, risk tolerance

[1722] Output: Financial data entered into the terminal

[1723] Step 2:

[1724] The terminal sends the user's input data to the server, which receives the data in real time and stores it in an internal database.

[1725] Input: Financial data entered into the terminal

[1726] Output: Financial data stored on the server

[1727] Step 3:

[1728] The server uses external APIs to retrieve additional financial data about the user, such as bank account details and investment portfolio, which is then integrated with the existing user data.

[1729] Input: Financial data stored on the server, plus additional financial data retrieved from external APIs

[1730] Output: Consolidated comprehensive user financial data

[1731] Step 4:

[1732] The server cleanses the consolidated financial data and corrects missing and outlier values, ensuring clean and standardized input data for generative AI models.

[1733] Input: Integrated comprehensive user financial data

[1734] Output: Clean, cleaned financial data

[1735] Step 5:

[1736] The server then inputs the cleansed data into a generative AI model, which has learned about past market trends and data from other users, and generates optimal financial advice for the user based on this data.

[1737] Input: Clean, cleaned financial data

[1738] Output: Customized financial advice from a generative AI model

[1739] Step 6:

[1740] The server then formats the generated financial advice in text and graph format and sends it back to the user's terminal, which receives it in real time and displays it in a format that is easy for the user to understand.

[1741] Input: Customized financial advice from a generative AI model

[1742] Output: Financial advice displayed on the terminal

[1743] Step 7:

[1744] The terminal provides tools for users to set savings goals and run investment simulations. Users can set specific savings goals and investment simulations and execute plans based on them.

[1745] Input: Financial advice displayed on user terminal

[1746] Output: Savings goals and investment simulation plans set by the user

[1747] Step 8:

[1748] Users can access financial advice and take concrete actions on savings and investments through their devices, and the app tracks progress in real time, enabling them to effectively manage their finances.

[1749] Input: Savings goals and investment simulation plans set by the user

[1750] Output: Administrative data on savings and investment actions taken

[1751] Through these steps, the present invention enables users to receive and implement effective, customized financial advice.

[1752] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1753] This invention is a system that allows users to input financial data and emotional data, and then provides customized financial advice based on that data using a generative AI model and an emotional engine. The following describes the processing for each server, terminal, and user in detail.

[1754] Server Processing

[1755] 1. Data reception:

[1756] The server first receives financial data (income, expenditure, asset status, risk tolerance, etc.) and emotional data (text messages, voice data, facial expression data, etc.) sent from the user's device. This data is sent to the server in real time and stored in an internal database.

[1757] 2. Data Integration:

[1758] The server uses external APIs to retrieve additional financial data about the user's bank account information and investment portfolio, and then integrates it with the received financial data. After data integration, it performs a cleansing process to correct missing values ​​and outliers.

[1759] 3. Emotion analysis:

[1760] The server uses an emotion engine to analyze the received emotion data, which identifies the user's emotional state (e.g., stress, relief, interest, etc.) and stores it in a database.

[1761] 4. Data Normalization and Analysis:

[1762] The cleansed data is standardized and fed into a generative AI model that takes into account market trends and past user data to generate optimal savings plans and investment strategies for users based on financial and sentiment analysis data.

[1763] 5. Advice generation and transmission:

[1764] The generated advice is formatted as text and graphs and sent to the user's device in an appropriate communication format (e.g., encouraging words or detailed explanations) based on the results of sentiment analysis.

[1765] Terminal handling

[1766] 1. Data Entry:

[1767] The terminal provides a form for users to input financial data and emotional data, including income, expenses, asset information, and risk tolerance, and emotional data including text messages, voice data, and facial expression data.

[1768] 2. Real-time validation and submission:

[1769] Once financial and emotional data is entered, the terminal validates it in real time, prompting the user to re-enter any incomplete data, and then sends the validated data to the server.

[1770] 3. Advice Receiving and Display:

[1771] The terminal receives customized financial advice sent from the server and converts it into an internal data model. The terminal visualizes the received advice and displays it in the form of a dashboard or report. Based on the results of sentiment analysis, the terminal provides advice to the user in an appropriate tone and format.

[1772] 4. Take action:

[1773] The device provides an interface for users to take specific actions based on the advice (such as setting savings goals or making investments), and the progress of the actions taken by the user is updated in real time on the device.

[1774] User operations

[1775] 1. Data Entry:

[1776] Users access the terminal and input their financial data (income, expenses, asset status, risk tolerance) and emotional data (text, voice, facial expressions).

[1777] 2. Data transmission:

[1778] Once the input is complete, the user presses the submit button to send the data to the server, and a notification message is displayed when the data is sent.

[1779] 3. Advice confirmation:

[1780] The user checks the customized financial advice sent from the server along with additional information based on sentiment analysis. For example, if the advice is to "save 1 million yen a year and invest 500,000 yen in a risk-diversified fund," an encouraging message and explanation of the risks will be added based on the results of sentiment analysis.

[1781] 4. Action execution:

[1782] Users can then take specific financial actions based on the advice, such as setting up regular savings or purchasing investment products, via their device.

[1783] 5. Feedback and Replanning:

[1784] The user frequently checks the progress of the action on the device, replans if necessary, and receives feedback based on their emotional state to help generate advice for the next action.

[1785] Specific examples

[1786] For example, User B has an annual income of 8 million yen, monthly living expenses of 250,000 yen, and a high risk tolerance. User B enters this financial data and emotional data, such as "I've been feeling stressed lately," into his / her device and sends it to the server. The server receives this data and analyzes it using a generative AI model and an emotion engine. As a result, it generates advice that is optimal for User B, such as "invest 1.5 million yen per year in a risk-diversified fund and put 500,000 yen into short-term savings." Furthermore, based on the results of the emotion analysis, it also provides "suggestions for low-risk investment strategies to reduce stress" and "detailed explanations to give a sense of security about investing."

[1787] In this way, the present invention provides customized financial advice that takes into account a user's overall financial situation and emotional state, helping users manage their finances more effectively.

[1788] The processing flow will be explained below.

[1789] Server processing flow

[1790] Step 1:

[1791] The server receives financial data and emotional data in real time from the user terminal. The financial data includes income, expenditure, asset status, and risk tolerance, and the emotional data includes text messages, voice data, and facial expression data.

[1792] Step 2:

[1793] The server uses external APIs to obtain additional financial data about the user's bank account details and investment portfolio, which is then stored in an internal database.

[1794] Step 3:

[1795] The server integrates the received financial data with the additional financial data it acquires, and performs a cleansing process to correct missing or outlier values.

[1796] Step 4:

[1797] The server uses an emotion engine to analyze the received emotion data, extracting emotion tags (e.g., stress, relief, interest, etc.) from text messages and voice data.

[1798] Step 5:

[1799] The server standardizes the cleansed financial data and feeds it, along with sentiment analysis results, into the generative AI model. The standardization process includes scaling numerical data and encoding categorical data.

[1800] Step 6:

[1801] The generative AI model analyzes the input data and generates optimal savings plans and investment strategies for users, and the analysis results are converted into text and graph formats.

[1802] Step 7:

[1803] The server then adds a message with an appropriate tone based on the results of sentiment analysis to the generated advice. For example, if a user is under stress, the server might recommend "low-risk investments" and add words of reassurance.

[1804] Step 8:

[1805] The server then sends the prepared advice to the user's device, securely using the HTTPS protocol.

[1806] Terminal processing flow

[1807] Step 1:

[1808] The terminal displays a form for users to enter financial and emotional data, including income, expenses, asset information, risk tolerance, and emotional data (text messages, voice, and facial expressions).

[1809] Step 2:

[1810] As users enter data into forms, the terminal validates the input data in real time and prompts the user to re-enter the data if it is incomplete.

[1811] Step 3:

[1812] The validated data is sent to the server securely using the HTTPS protocol.

[1813] Step 4:

[1814] The terminal receives customized financial advice sent from the server and converts the received data into an internal data model.

[1815] Step 5:

[1816] The device visualizes and displays advice to the user in the form of a dashboard or report, and delivers messages in an appropriate tone based on the results of sentiment analysis.

[1817] Step 6:

[1818] Providing an interface for users to take specific actions based on advice, including interfaces to help set savings goals and make investments.

[1819] Step 7:

[1820] The progress of actions performed by the user is updated in real time and displayed on the device.

[1821] User Process Flow

[1822] Step 1:

[1823] Users access the terminal and input their financial data (income, expenses, asset status, risk tolerance) and emotional data (text messages, voice, facial expressions).

[1824] Step 2:

[1825] Once the input is complete, the user presses the submit button to send the data to the server, and a notification message is displayed when the data has been sent.

[1826] Step 3:

[1827] The user checks the customized financial advice sent from the server and messages based on sentiment analysis. For example, the advice might be "save 1 million yen a year and invest 500,000 yen in a risk-diversified fund," along with suggestions for low-risk investment strategies to reduce stress and encouraging messages.

[1828] Step 4:

[1829] Users take specific financial actions based on the advice, such as setting up regular savings or purchasing investment products.

[1830] Step 5:

[1831] Users frequently check the progress of their actions on their devices and make replanning and adjustments as needed.

[1832] Step 6:

[1833] Users can provide feedback on their achieved goals and ongoing plans via their devices, which will be used to generate the next advice. This feedback will be used to provide even more accurate advice.

[1834] Example 2

[1835] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1836] In modern financial management, not only individual users' financial data but also their emotional state at any given time is an important factor. However, conventional financial advisory systems have been unable to fully utilize emotional data, making it difficult to provide customized advice. This has led to issues such as users feeling stressed when making appropriate financial management and investment decisions, making it difficult to manage their finances efficiently.

[1837] The identification process by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means. In this invention, the server includes: means for inputting financial data and emotion data from a user; means for transmitting the financial data and emotion data to the server; means for using a generative AI model and an emotion engine to analyze the data; means for generating customized financial advice using the generative AI model and emotion engine; means for transmitting the customized financial advice to a user terminal; means for displaying the customized financial advice on the user terminal; means for the server to obtain additional financial data using an external API and integrate the data; means for the server to analyze the emotion data and identify the user's emotional state; means for the user terminal to validate the financial data and emotion data in real time and notify the user of any input errors; means for the customized financial advice to be provided to the user in a form that reflects the emotion analysis results; means for the user terminal to support the user in taking action based on the advice; and means for the user to provide feedback that is used to generate the next advice. This enables more personalized financial management for users and provides optimal advice that takes into account their emotional state and market trends.

[1838] markdown

[1839] A "user" is an individual or entity that uses the system to input financial and emotional data and receive customized financial advice.

[1840] "Financial data" refers to data relating to the user's economic situation, such as income, expenses, asset status, and risk tolerance.

[1841] "Emotion data" refers to data such as text messages, voice data, and facial expression data that indicate the user's emotional state.

[1842] A "server" is a central computing unit that receives data sent from user terminals, performs analysis, and generates financial advice.

[1843] A "generative AI model" is a machine learning model that takes into account market trends and historical data to generate customized financial advice based on input data.

[1844] An "emotion engine" is a software module for analyzing emotion data and identifying the user's emotional state.

[1845] A "user terminal" is an electronic device through which a user inputs financial and emotional data and receives customized financial advice transmitted from a server.

[1846] An "external API" is an interface for obtaining data from other systems or services.

[1847] "Real-time validation" is a process that immediately checks input data and prompts correction of any incomplete data.

[1848] "Customized financial advice" is advice generated by a generative AI model and emotion engine based on a user's individual financial situation and emotional state.

[1849] A "means for supporting behavioral execution" is a part of the system that provides an interface or functionality for users to take specific actions based on customized financial advice.

[1850] "Feedback" refers to opinions and evaluations provided by users to the system, and is information that will be used to generate the next piece of advice.

[1851] MODE FOR CARRYING OUT THE INVENTION

[1852] The present invention is a system for providing optimal financial advice to users based on financial data and emotional data. The processing performed by the server, terminal, and user will be described in detail below.

[1853] Server Processing

[1854] Data reception

[1855] The server first receives financial data (income, expenses, asset status, risk tolerance, etc.) and emotional data (text messages, voice data, facial expression data, etc.) sent from the user's device. This data is sent to the server in real time and stored in an internal database. For example, if a user sends data from their device such as "monthly income of 500,000 yen, monthly expenses of 300,000 yen," this information is stored in the receiving buffer.

[1856] Data Integration

[1857] The server uses external APIs to retrieve additional financial data about the user's bank account information and investment portfolio, and then integrates it with the original financial data. This process also includes a cleansing process to automatically correct missing data and outliers. For example, if a user has multiple accounts and their account information is retrieved via an API, the collected data can be integrated with the input data.

[1858] Emotion analysis

[1859] The server uses an emotion engine to analyze the received emotion data. Through this analysis, the user's emotional state (e.g., stress, relief, interest, etc.) is identified and stored in a database. For example, from a user's message such as "I'm tired today," the emotion engine determines the user's stress level.

[1860] Data normalization and analysis

[1861] The cleansed data is then standardized and fed into a generative AI model, which takes into account market trends and historical data to generate optimal savings plans and investment strategies for users based on financial and sentiment data. For example, historical market data could be used to suggest optimal investment options for users.

[1862] Advice generation and delivery

[1863] The generated advice is then formatted as text and graphs and sent to the user in an appropriate communication format based on the results of the emotion analysis. For example, advice containing encouraging words and detailed explanations may be sent.

[1864] Terminal handling

[1865] Data Entry

[1866] The device provides a form for users to input financial and emotional data. Financial data includes income, expenses, asset information, and risk tolerance, while emotional data includes text messages, voice data, and facial expression data. For example, the device has text boxes for inputting "income" and "expenses," as well as voice recording and facial expression recognition functions for expressing "emotions."

[1867] Real-time validation and submission

[1868] When financial and emotional data is entered, the terminal validates it in real time and displays a warning if the data is incomplete. For example, if a negative income value is entered, an error message is displayed prompting the user to correct it. The terminal also sends the validated data to the server.

[1869] Advice reception and display

[1870] The device receives customized financial advice sent from the server and displays it visually, including graphical dashboards and detailed text messages. For example, if the advice is to "save 1 million yen a year and invest 500,000 yen in a risk-diversified fund," it will visualize it in pie charts and bar graphs.

[1871] Action Execution

[1872] The device provides an interface for users to take action based on the advice. For example, a "Set Savings Goal" button or an "Invest" button are displayed, and users can tap the button to start taking specific action.

[1873] User operations

[1874] Data Entry

[1875] The user uses the device to input their own financial data (income, expenditure, asset status, risk tolerance) and emotional data (text, voice, facial expression). For example, they input "income 500,000 yen, expenditure 300,000 yen" and also emotional data such as "I've been under a lot of stress lately."

[1876] Data transmission

[1877] The data you have entered will be sent to the server. When you press the send button, the progress will be displayed and you will be notified when the data has been sent.

[1878] Advice confirmation

[1879] Check the customized advice sent from the server. For example, "Invest 1.5 million yen a year in a risk-diversified fund and put 500,000 yen into short-term savings," along with an encouraging message based on the results of sentiment analysis.

[1880] Action execution

[1881] The user then takes specific actions based on the advice, such as purchasing an investment fund, and checks the progress of the action on the device.

[1882] Feedback and replanning

[1883] Users can track the progress of their actions, reorganize their plans if necessary, and provide feedback on their emotional state to help inform the next recommendation. For example, they can provide feedback like, "I'm happy with my current investments, but I'd like to see a less risky option," which will be reflected in the next recommendation.

[1884] Specific examples

[1885] For example, User B has an annual income of 8 million yen, monthly living expenses of 250,000 yen, and a high risk tolerance. User B enters this financial data and emotional data, such as "I've been feeling stressed lately," into his / her device and sends it to the server. The server analyzes the received data and generates advice such as "invest 1.5 million yen per year in a risk-diversified fund and put the remaining 500,000 yen into short-term savings." Furthermore, based on the results of the emotional analysis, the advice is accompanied by "low-risk investment strategies to reduce stress" and "detailed explanations to give a sense of security about investing." This advice is displayed on the device, and the user then takes specific action (purchase an investment fund) through the device.

[1886] Prompt Sentence Examples

[1887] "Enter information about your income and expenses, financial situation, and risk tolerance, and provide your recent emotional state via text, voice, and facial expressions."

[1888] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1889] System program processing flow

[1890] Server Processing

[1891] Step 1: Receiving data

[1892] The server receives financial data (income, expenditure, asset status, risk tolerance) and emotional data (text messages, voice data, facial expression data) sent from the user's device. This data is stored in a receiving buffer in real time. For example, information entered by a user as income of 500,000 yen and expenditure of 300,000 yen is saved.

[1893] Input: Financial and emotional data from user devices

[1894] Output: Data stored in the receive buffer

[1895]

[1896] Step 2: Data Integration

[1897] The server uses external APIs to retrieve additional financial data about the user's bank account information and investment portfolio, and combines it with the incoming financial data. This combined data undergoes cleansing processes to correct missing and outlier values. For example, data retrieved from financial institution APIs can be combined with user input data.

[1898] Input: Incoming financial data and financial data obtained from external APIs

[1899] Output: Unified and cleansed data

[1900]

[1901] Step 3: Sentiment Analysis

[1902] The server uses an emotion engine to analyze the received emotion data. This analysis identifies the user's emotional state (e.g., stress, relief, interest) and stores it in a database. For example, if a user says, "I'm tired today," the emotion engine analyzes the user's stress level.

[1903] Input: Received emotion data

[1904] Output: Parsed emotional state data

[1905]

[1906] Step 4: Data standardization and analysis

[1907] The server standardizes the cleansed data and inputs it into a generative AI model, which takes into account market trends and historical data, and generates optimal savings plans and investment strategies for users based on financial and sentiment data. For example, it might suggest optimal investments based on user data combined with market trend data.

[1908] Input: Cleansed and standardized data

[1909] Output: Analysis results from the generative AI model (savings plan and investment strategy)

[1910]

[1911] Step 5: Advice generation and sending

[1912] The generated advice is then formatted as text and graphs and sent to the user in an appropriate communication format based on the results of the emotion analysis. For example, advice containing encouraging words and detailed explanations may be sent.

[1913] Input: Analysis results and emotional state data

[1914] Output: Customized financial advice sent to the user

[1915] Terminal handling

[1916] Step 1: Data entry

[1917] The device provides a form for users to enter financial and emotional data, including text boxes for entering income and expenses, as well as voice recording and facial recognition capabilities.

[1918] Input: User financial and emotional data input

[1919] Output: Data entered in the input form

[1920]

[1921] Step 2: Real-time validation and submission

[1922] The terminal validates the data entered in real time and displays a warning if the data is incomplete. For example, if a negative income value is entered, an error message is displayed and the user is prompted to correct the data. After validation is complete, the data is sent to the server.

[1923] Input: User-entered financial and sentiment data

[1924] Output: Send validated data to the server

[1925]

[1926] Step 3: Receive and view advice

[1927] The terminal receives customized financial advice sent from the server and displays it in a visualized format (dashboard and graphs). For example, advice such as "save 1 million yen a year and invest 500,000 yen in a risk-diversified fund" is visualized.

[1928] Input: Customized financial advice from the server

[1929] Output: Visualized advice display

[1930]

[1931] Step 4: Take Action

[1932] The device provides an interface for users to take specific actions based on the advice, such as a "Set Savings Goal" button or an "Invest" button that users can tap to start the action.

[1933] Enter: customized financial advice

[1934] Output: User action execution

[1935] User operations

[1936] Step 1: Data entry

[1937] The user inputs financial data (income, expenditure, asset status, risk tolerance) and emotional data (text, voice, facial expression) into the terminal. For example, the user inputs "income 500,000 yen, expenditure 300,000 yen" and emotional data such as "I've been under a lot of stress lately."

[1938] Input: Financial and sentiment data

[1939] Output: Filling in the input form

[1940]

[1941] Step 2: Send data

[1942] The data you have entered will be sent to the server. When you press the send button, the progress will be displayed and you will be notified when the data has been sent.

[1943] Input: Data entered into an input form

[1944] Output: Send data to the server

[1945]

[1946] Step 3: Check the advice

[1947] The user then checks the customized financial advice sent from the server, such as "Invest 1.5 million yen a year in a risk-diversified fund and put 500,000 yen into short-term savings," along with an encouraging message based on the results of sentiment analysis.

[1948] Input: Customized financial advice sent from the server

[1949] Output: Check the visualized advice

[1950]

[1951] Step 4: Take action

[1952] The user then takes specific actions based on the advice, such as purchasing an investment fund, and checks the progress of the action on the device.

[1953] Enter: customized financial advice

[1954] Output: Implementing specific actions

[1955]

[1956] Step 5: Feedback and replanning

[1957] Users can track their progress and adjust their plans as needed. The system also provides feedback based on their emotional state, such as, "I'm happy with my current investments, but I'd like to see a less risky option."

[1958] Input: Action execution results and emotional feedback

[1959] Output: Data used to generate next advice

[1960]

[1961] (Application example 2)

[1962] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1963] In conventional factory work, workers' emotional states have a significant impact on work performance and safety measures, but there is a lack of effective ways to manage them in real time. Furthermore, there is a lack of technology to provide optimal advice to each individual worker. Therefore, there is a need to improve work performance and strengthen safety measures.

[1964] The specific processing by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for inputting financial data and emotional data from a user, means for transmitting the financial data and emotional data to the server, means for using a generative AI model and an emotion engine that analyzes the financial data and emotional data, means for generating customized financial advice and work performance improvement advice using the generative AI model and emotion engine, means for transmitting the customized financial advice and work performance improvement advice to a user terminal, and means for displaying the customized financial advice and work performance improvement advice on the user terminal. This makes it possible to analyze the emotional state of a worker in real time and provide optimal advice.

[1965] "User" refers to a person or organization that uses the system.

[1966] "Financial Data" refers to financial information about an individual or organization, such as income, expenses, assets, and risk tolerance.

[1967] "Emotional data" refers to information that represents an emotional state, such as text messages, voice data, and facial expression data.

[1968] "Server" refers to the central processing unit that receives and analyzes data sent by the User and retransmits the generated information to the User.

[1969] "Generative AI model" refers to a machine learning model that analyzes a user's financial and emotional data to generate customized advice.

[1970] "Emotion engine" refers to an analysis system for analyzing emotion data and identifying a user's emotional state.

[1971] "User terminal" refers to a device used by a user to input data or receive and display information sent from the server.

[1972] "External Data Acquisition Means" refers to means for acquiring and integrating additional information from external data sources.

[1973] "Work data" refers to data related to the progress and performance of work in a factory or on-site.

[1974] "Operation data" refers to data relating to the operating status of machinery and equipment within a factory.

[1975] The present invention is a system that utilizes a user's financial data and emotion data to provide customized financial advice, including work performance improvement and safety measures, using a generative AI model and emotion engine. Specific processing for each server, terminal, and user is described below.

[1976] Server Processing

[1977] 1. Data Reception

[1978] The server receives financial data (e.g., income, expenses, asset status, risk tolerance) and emotional data (e.g., text messages, voice data, facial expression data) sent from the user's device. This data is sent to the server in real time and stored in an internal database.

[1979] 2. External Data Acquisition and Data Integration

[1980] The server uses external data acquisition means to acquire work data and equipment operation data. The acquired data is integrated and cleansed. This process uses data integration programs and data cleaning algorithms in a Python environment.

[1981] 3. Sentiment Analysis and Data Analysis

[1982] The server analyzes the emotion data using an emotion engine (e.g., Spacy, IBM Watson). The analysis results are combined with financial data and input into a generative AI model (e.g., TensorFlow, PyTorch). The generative AI model uses this data to generate customized financial advice and work performance improvement advice.

[1983] 4. Advice Generation and Delivery

[1984] The generated advice is formatted as text and graphs and sent to the user's device in an appropriate communication format (e.g., detailed explanations that provide a sense of security) based on the results of sentiment analysis.

[1985] Terminal handling

[1986] 1. Data Entry

[1987] The device provides an interface for users to input financial and emotional data, including text fields, voice input, and facial expression input using a camera.

[1988] 2. Data Transmission

[1989] The entered data is sent from the terminal to the server. This process includes a data transmission program and validation check function.

[1990] 3. Receiving and displaying advice

[1991] The customized advice sent from the server is received by the device, visualized, and displayed to the user. This user interface includes dashboards and reports. In particular, the advice for improving work performance is displayed on smart glasses.

[1992] User operations

[1993] 1. Data Entry

[1994] The user accesses the input interface of the terminal and inputs their own financial data (e.g., income, expenses, asset status, risk tolerance) and emotional data (e.g., text, voice, facial expression).

[1995] 2. Data Transmission

[1996] Once the input is complete, the user presses the submit button to send the data to the server, and a notification message is displayed when the data is sent.

[1997] 3. Check and implement the advice

[1998] After checking the customized advice sent from the server, the user can take specific actions based on the advice, for example, a worker can adjust their work performance or take a break according to the advice displayed on the smart glasses.

[1999] Specific examples

[2000] For example, if Worker A working in a factory comments that he has been under a lot of stress lately, and this emotional state is also confirmed by facial expression data, the server will generate advice to Worker A based on this, recommending that he prioritize low-risk work and take a temporary break. This advice is displayed on the smart glasses.

[2001] Prompt Sentence Examples

[2002] "Design a real-time emotion analysis application for factory workers. This application requires workers to input their emotion data (e.g., stress or relief) and work performance data, and then displays appropriate advice in real time on smart glasses. Data analysis is based on a generative AI model and an emotion engine. Please also generate the corresponding Python code."

[2003] The present invention is a system that handles emotional data and financial data in an integrated manner to provide customized advice to factory workers, thereby improving productivity and strengthening safety measures.

[2004] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[2005] Step 1: The user inputs financial and emotional data into the terminal.

[2006] Users use the device's interface to input financial data such as income, expenses, asset status, and risk tolerance. They also input text messages, voice data, and facial expression data that indicate their emotional state via a camera or microphone. The input data is temporarily stored in the device.

[2007] Step 2: The terminal performs real-time validation and sends the data to the server.

[2008] The terminal validates the entered financial and emotional data in real time to check for incomplete data or outliers. Once validated, the data is encrypted and sent to the server, where it checks the input data (income, expenses, emotional state) and either displays an error message or converts it into a format that can be sent.

[2009] Step 3: The server receives the data and stores it in the database.

[2010] The server receives the financial and emotional data sent from the device and stores it in an internal database, which is used in subsequent processing steps.

[2011] Step 4: The server acquires the additional data using an external data acquisition means.

[2012] The server makes API calls to retrieve work data and equipment operation data from external sources. The retrieved data is integrated with internal data and stored in a database. Additional data (such as equipment operation status) is retrieved and integrated with the existing database.

[2013] Step 5: The server performs the cleansing process.

[2014] The server performs a cleansing process on all the acquired data, filling in missing values ​​and correcting outliers, using a Python data analysis library.

[2015] Step 6: The server performs sentiment analysis.

[2016] The server analyzes the emotional data using an emotion engine (e.g., Spacy, IBM Watson). The analysis results are stored in a database as the user's emotional state (stress, relief, etc.). Input: Emotional data (text, voice, facial expression), Output: Emotional state (stress, elation, etc.).

[2017] Step 7: The server inputs the data into the generative AI model and performs analysis.

[2018] The cleansed data and sentiment analysis results are input into a generative AI model. The generative AI model (e.g., TensorFlow, PyTorch) analyzes the financial data and integrated data to generate customized financial and work performance improvement advice. Input: Financial data, sentiment data, work data. Output: Customized advice (savings plans, investment strategies, work instructions, etc.).

[2019] Step 8: The server sends the generated advice.

[2020] The generated advice is formatted as text and graphs and sent to the user terminal in an appropriate communication format. Input: Advice data (text, graphs), Output: Transmission to the user terminal.

[2021] Step 9: The device receives and displays the advice.

[2022] The user device receives the customized advice sent from the server and visualizes and displays it. The display format is a dashboard or report. In particular, advice for improving work performance is displayed on smart glasses, etc. Input: Received data (advice), Output: Visual display (graph, text).

[2023] Step 10: The user acts on the advice.

[2024] The user checks the customized advice displayed on the device and takes specific actions based on it, such as creating a savings plan or changing the priorities of tasks. Input: Check the advice, Output: Specific actions (savings, investment, task change, etc.).

[2025] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the control target 443 to output the result of the specific processing. The microphone 238 acquires voice indicating a user input regarding the result of the specific processing. The control unit 46A transmits voice data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the voice data.

[2026] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[2027] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the robot 414.

[2028] The emotion identification model 59 as an emotion engine may determine the user's emotion according to a specific mapping. Specifically, the emotion identification model 59 may determine the user's emotion according to an emotion map (see FIG. 9), which is a specific mapping. Similarly, the emotion identification model 59 may determine the robot's emotion, and the identification processing unit 290 may perform identification processing using the robot's emotion.

[2029] FIG. 9 is a diagram illustrating an emotion map 400 on which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. Emotions closer to the center of the concentric circles are more primitive. Emotions representing states and actions arising from a state of mind are arranged on the outer edges of the concentric circles. The concept of emotion includes both affect and mental states. Emotions generally generated from reactions occurring in the brain are arranged on the left side of the concentric circles. Emotions generally induced by situational judgment are arranged on the right side of the concentric circles. Emotions generally generated from reactions occurring in the brain and induced by situational judgment are arranged on the upper and lower sides of the concentric circles. Furthermore, the emotion of "pleasure" is arranged on the upper side of the concentric circles, and the emotion of "discomfort" is arranged on the lower side. In this way, in the emotion map 400, multiple emotions are mapped based on the structure by which emotions are generated, and emotions that tend to occur simultaneously are mapped close to each other.

[2030] These emotions are distributed in the 3 o'clock direction on emotion map 400, and typically fluctuate between relief and anxiety. In the right half of emotion map 400, situational awareness dominates over internal sensations, resulting in a sense of calm.

[2031] The inside of emotion map 400 represents what is going on in the mind, and the outside of emotion map 400 represents behavior, so the further you go outside emotion map 400, the more visible the emotions become (the more they are expressed in behavior).

[2032] Human emotions are based on various balances, such as posture and blood sugar levels. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. Emotions can also be created for robots, automobiles, and motorcycles, based on various balances, such as posture and remaining battery life. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ...

Claims

1. a means for inputting financial data from a user; means for transmitting said financial data to a server; means for using a generative AI model to analyze the financial data; means for generating customized financial advice using the generative AI model; means for transmitting the customized financial advice to a user terminal; means for displaying the customized financial advice at the user terminal; A system including:

2. The system of claim 1 , wherein the server includes means for utilizing an external API to obtain additional financial data.

3. The system of claim 1 , wherein the generative AI model includes means for performing analysis by considering historical data and market trends.

Images