Information processing device, information processing method, and program

Abstract

To improve user engagement with electronic payment services. [Solution] An information processing device comprising: an acquisition unit that acquires a remittance request to a second user entered by a first user of an electronic payment service; a display control unit that, in response to the acquisition of the remittance request, displays an option on the first user's first user terminal device for selecting whether or not to generate an image to be sent to the second user's second user terminal device; a generation unit that generates an image to be sent to the second user terminal device in response to the first user's selection; and a transmission unit that transmits the generated image to the second user terminal device.

Description

Technical Field

[0001] The present invention relates to an information processing apparatus, an information processing method, and a program.

Background Art

[0002] Conventionally, in electronic payment services, a technology that enables communication and money transfer between users using peer-to-peer (P2P) communication is known. For example, Patent Document 1 describes a technology that enables a user to select a money transfer after knowing the amount to be paid to another user when multiple users pay a fee.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] However, in the prior art, generally, in addition to the execution of money transfer, no added value as a customer experience was given to the users of the money transfer function. As a result, the engagement of users with the electronic payment service may have been low.

[0005] The present invention has been made in consideration of such circumstances, and one of its objectives is to provide an information processing apparatus, an information processing method, and a program that can improve the engagement of users with the electronic payment service.

Means for Solving the Problems

[0006] One aspect of the present invention is an information processing device comprising: an acquisition unit that acquires a remittance request to a second user input by a first user of an electronic payment service; a display control unit that, in response to the acquisition of the remittance request, displays options on the first user's first user terminal device for selecting whether or not to generate an image to be transmitted to the second user's second user terminal device; a generation unit that generates an image to be transmitted to the second user terminal device in response to the first user's selection; and a transmission unit that transmits the generated image to the second user terminal device. [Effects of the Invention]

[0007] According to one aspect of the present invention, it is possible to improve user engagement with electronic payment services. [Brief explanation of the drawing]

[0008] [Figure 1] This diagram shows an example of a configuration for implementing an electronic payment service. [Figure 2] This is a sequence diagram (part 1) illustrating the general flow of electronic payments. [Figure 3] This is a sequence diagram (part 2) illustrating the general flow of electronic payments. [Figure 4] This is a configuration diagram of the payment server 100 according to the first embodiment. [Figure 5] This figure shows an example of the contents of user information 172. [Figure 6] This figure shows an example of the contents of merchant / store information 176. [Figure 7] This figure shows an example of the top screen of a payment app 20. [Figure 8] This figure shows a scene where a user launches the AI ​​assist app 20A and enters a request in natural language. [Figure 9] This figure shows an example of an orchestrator prompt 178 that the orchestrator unit 150A inputs to the LLM server 200. [Figure 10]FIG. is an example of an agent prompt 180 input by the agent unit 150B to the LLM server 200. [Figure 11] FIG. is an example of a common UI component defined in the UI component information 186. [Figure 12] FIG. is a screen transition diagram showing the input task of the remittance recipient in the remittance process. [Figure 13] FIG. is a screen transition diagram showing the input task of the remittance amount in the remittance process. [Figure 14] FIG. is a screen transition diagram showing the additional task of the memo in the remittance process. [Figure 15] FIG. is a screen transition diagram showing the final confirmation task before the execution of the remittance process. [Figure 16] FIG. is a screen transition diagram for notifying the user that the remittance process has been completed. [Figure 17] FIG. is a screen transition diagram showing the input task of the charge amount in the charge process. [Figure 18] FIG. is a screen transition diagram showing the final confirmation task before the execution of the charge process. [Figure 19] FIG. is a screen transition diagram for notifying the user that the charge process has been completed. [Figure 20] FIG. is an example of a diagram showing the flow of processing when an error occurs. [Figure 21] FIG. is a sequence diagram showing an example of the flow of processing of the AI assist function. [Figure 22] FIG. is an example of a diagram showing a screen transition involving image generation in the remittance process using the AI assist function. [Figure 23] FIG. is an example of a diagram showing a screen transition involving image generation in the remittance process using the AI assist function. [Figure 24] FIG. is an example of a diagram showing a screen transition involving image generation in the normal remittance process. [Figure 25] FIG. is an example of a diagram showing a screen transition involving image generation in the normal remittance process. [Figure 26] FIG. is an example of a diagram showing a screen transition involving image generation in the normal remittance process.

Best Mode for Carrying Out the Invention

[0009] Hereinafter, embodiments of an information processing apparatus, an information processing method, and a program of the present invention will be described with reference to the drawings. Various apparatuses such as "servers", "management apparatuses", and "information providing apparatuses" that provide services to users or perform internal analysis may be realized by a decentralized group of apparatuses, and the operators of each apparatus may be different. Also, the holder of the hardware of the apparatus (provider of the cloud server) and the operator who actually performs the operation may be different. The application program and the payment server cooperate to provide an electronic payment service. In the following description, the application program is referred to as a payment application. The electronic payment service is a service that supports payment related to the purchase of goods and services in a store. A store is, for example, a physical store (actual store) existing in the real space, but may include a virtual store for e-commerce. The virtual store may include those provided by a subject different from the operator of the electronic payment service. In that case, when making a payment for shopping in the virtual store, it may be controlled to transition to the interface screen of the electronic payment service. In the electronic payment service, a store is, for example, treated as belonging to a franchise (brand), and processing such as payment when a purchase action is performed in the store is mainly carried out between the user and the franchise. Instead of this, processing such as payment may be carried out between the user and the store.

[0010] [Electronic Payment Service] FIG. 1 is a diagram showing an example of a configuration for realizing an electronic payment service. The electronic payment service is realized centering around a payment server 100. The payment server 100 communicates with each of, for example, one or more user terminal devices 10, one or more first store terminal devices 50, and one or more second store terminal devices 70 via a network NW. The network NW includes, for example, the Internet, a LAN (Local Area Network), a wireless base station, a provider device, and the like.

[0011] The user terminal device 10 is, for example, a portable terminal device such as a smartphone or tablet. The user terminal device 10 is a computer device having at least optical reading function, communication function, display function, input reception function, and program execution function. In the following description, the components for realizing these functions will be referred to as a camera, communication device, touch panel, CPU (Central Processing Unit), etc. In the user terminal device 10, the payment application 20 is executed by a processor such as the CPU, and it operates in cooperation with the payment server 100 to provide electronic payment services to the user. The payment application 20 is installed on the user terminal device 10, for example, from an application store, and controls the camera, communication device, touch panel, etc.

[0012] In this embodiment, the payment application 20 incorporates an AI assist application 20A. The AI ​​assist application 20A accepts natural language input from the user and, in cooperation with the AI ​​assist function unit 150 of the payment server 100 (described later), provides a function to perform processing in a conversational format in response to the user's request. The AI ​​assist application 20A may be a thin client application that displays input / output information from the AI ​​assist function unit 150 (described later), or it may include some or all of the functions of the AI ​​assist function unit 150. Alternatively, the AI ​​assist application 20A may function as a web browser application that displays input / output information from the AI ​​assist function unit 150.

[0013] The first store terminal device 50 is installed, for example, in a store. The first store terminal device 50 is a computer device having at least a product price acquisition function, an optical reading function, a program execution function, and a communication function. The first store terminal device 50 may include a so-called POS (Point of Sale) device, and the product price acquisition function and optical reading function may be realized by the POS device. The store code image 60 is placed in the store and is a code image such as a QR code (registered trademark) printed on paper or plastic media. The store code image 60 may also be displayed on a display placed in the store (which may be the display of a terminal device such as a smartphone).

[0014] The second store terminal device 70 is used by the operator of the affiliated store. The second store terminal device 70 is a smartphone, tablet, personal computer, etc. The affiliated store interface 72 operates on the second store terminal device 70. The affiliated store interface 72 may be an affiliated store application or a browser. The affiliated store interface 72 accepts coupon settings etc. from the affiliated store operator and transmits them to the payment server 100. The second store terminal device 70, which is a smartphone, has the function of displaying a code image corresponding to a store code image by running the affiliated store application, or reading a code image displayed by the user terminal device 10.

[0015] The payment server 100 implements electronic payment based on payment information received from the user terminal device 10 or the first store terminal device 50. The first store terminal device 50 may include a POS device and a merchant server, in which case payment information is transmitted from the POS device to the payment server 100 via the merchant server. In the following description, these will not be specifically distinguished, and it will be assumed that payment information is transmitted from the first store terminal device 50. The payment server 100 also communicates with the LLM server 200 via the network NW.

[0016] The LLM server 200 is a server device such as a web server. The LLM server 200 is equipped with a large language model (LLM) that has been trained to receive user input information and prompts as text from the payment server 100 and generate inference results as text according to the received content. More specifically, the LLM server 200 interprets the user's intent in response to a request from the orchestrator unit 150A (described later) and determines which agent unit 150B should execute the processing. The LLM server 200 also breaks down the processing that the agent unit 150B should execute into multiple tasks in response to a request from the agent unit 150B, determines the UI components to be used in each task, and notifies the agent unit 150B. In this embodiment, the LLM server 200 is installed outside the payment server 100, but the present invention is not limited to such a configuration, and the LLM server 200 may be part of the functions of the payment server 100 (for example, the AI ​​assist function unit 150). The large-scale language model installed in the LLM server 200 is an example of "LLM". For the sake of brevity of explanation, even if a process is executed by the LLM server 200, if it is a process requested by the orchestrator unit 150A or the agent unit 150B, these orchestrator unit 150A or agent unit 150B may be referred to as the subject (i.e., the operator) in the description.

[0017] Figures 2 and 3 are sequence diagrams illustrating the general flow of electronic payments. There may be two patterns for electronic payments: Pattern 1 and Pattern 2.

[0018] In the case of Pattern 1 shown in Figure 2 (hereinafter referred to as User Scan), the user terminal device 10, with the payment application 20 running, decodes the store code image 60 using its optical reading function (S1). The store code image 60 contains information about the store URL (Uniform Resource Locator). This store URL is an electronic payment service domain to which information that can identify the store has been added, and is associated with the merchant ID and store ID, etc., at the payment server 100 (described later). The payment application 20 sends the first payment information, including the store URL and account ID, to the payment server 100 (S2). The payment server 100 searches for store information (described later) from the merchant ID and store ID corresponding to the store URL, obtains the merchant name and store name information (S3), and sends it to the payment application 20 (S4). The user enters the payment amount into the user terminal device 10 on the screen where the merchant name and store name are displayed (S5). The user terminal device 10 then generates second payment information, including at least the payment amount, and sends it to the payment server 100 (S6). The payment server 100 performs electronic payment based on the received second payment information (S7). The payment server 100 then sends a payment completion notification (information for displaying the payment completion screen) to the payment application 20 (S8), and the payment application 20 displays the payment completion screen (S9). If the store code image 60 is displayed on a display placed in the store, the store code image 60 may include payment amount information as well as the store URL. In this case, the procedure for the user to enter the payment amount is omitted, and the payment amount information is included in the first payment information and sent to the payment server 100. Merchant name and store name information may be included and displayed on the payment completion screen.

[0019] In the case of Pattern 2 shown in Figure 3 (hereinafter referred to as Store Scan), when the payment app 20 is launched, when a payment operation is performed in the payment app 20, when it is time for an automatic update (for example, every minute), and at other times, the payment app 20 sends a request to the payment server 100 to issue a one-time code (S11). The payment server 100 generates a one-time code (S12) and sends it to the payment app 20 (S13). The payment app 20 displays a code image such as a QR code or barcode that was generated based on the one-time code (S14). The user holds the display surface of the user terminal device 10 over the first store terminal device 50 (presents it), and the first store terminal device 50 decodes the code image using its optical reading function and obtains the one-time code, etc. (S15). Then, the first store terminal device 50 generates payment information including the one-time code, payment amount, merchant ID, store ID, etc., and sends it to the payment server 100 (S16). The payment amount information is obtained in advance by barcode scanning or manual input. Based on the received information, the payment server 100 identifies the user corresponding to the one-time code and performs the electronic payment (S17). The payment server 100 then sends a payment completion notification to the payment app 20 (S18), and the payment app 20 displays a payment completion screen (S19).

[0020] Furthermore, electronic payment may be performed using only one of the above patterns. Also, the "account ID" explained in Figure 2 may be other information that can be used as user identification information (for example, a phone number). In addition, the issuance of a one-time code may be omitted during store scanning, and the payment app 20 may display a code image generated based on the user's account ID. In that case, the payment server 100 will identify the user corresponding to the account ID instead of identifying the user corresponding to the one-time code.

[0021] [Payment Server] Figure 4 is a configuration diagram of the payment server 100 according to the first embodiment. The payment server 100 includes, for example, a communication unit 110, a payment content provision unit 120, a payment processing unit 130, an information management unit 140, an AI assist function unit 150, and a storage unit 170. Components other than the communication unit 110 and the storage unit 170 are realized, for example, by a hardware processor such as a CPU executing a program (software). Some or all of these components may be realized by hardware (including circuitry) such as LSI (Large Scale Integration), ASIC (Application Specific Integrated Circuit), FPGA (Field-Programmable Gate Array), and GPU (Graphics Processing Unit), or by the cooperation of software and hardware. The program may be stored in advance on a storage device such as an HDD (Hard Disk Drive) or flash memory (a storage device equipped with a non-transient storage medium), or it may be stored on a removable storage medium such as a DVD or CD-ROM (a non-transient storage medium) and installed on the storage device when the storage medium is inserted into the drive device.

[0022] The storage unit 170 can be an HDD, flash memory, RAM (Random Access Memory), etc. The storage unit 170 may also be a NAS (Network Attached Storage) device accessible by the payment server 100 via the network. The storage unit 170 stores information such as user information 172, payment content information 174, and merchant / store information 176. In addition, the storage unit 170 stores orchestrator prompts 178, agent prompts 180, document information 182, API information 184, and UI component information 186, which are used by the AI ​​assist function unit 150. This information will be described later.

[0023] The communication unit 110 is a communication interface for connecting to a network NW. The communication unit 110 is, for example, a network interface card.

[0024] The payment content provision unit 120, for example, has the functionality of a web server and provides information (content) for displaying various screens of the electronic payment service to the user terminal device 10. The payment content provision unit 120 reads the necessary content from the payment content information 174 as appropriate and provides it to the user terminal device 10. The user terminal device 10 receives various inputs from the user while the content is being played by the payment application 20 and transmits the aforementioned payment information and other data to the payment server 100.

[0025] The payment processing unit 130 performs payment processing based on payment information transmitted by the user terminal device 10 or the first store terminal device 50. The payment processing unit 130 performs payment processing while referring to the user information 172.

[0026] Figure 5 shows an example of the contents of User Information 172. User Information 172 is an example of user registration information. User Information 172 includes, for example, user URL, account ID, telephone number, password, as well as information such as email address, user ID, name, address, date of birth, registration date, charge balance, credit payment settings, credit payment limit, credit payment amount, available credit payment amount, payment method settings, bank account, credit card number, charge history information, and payment history information. The user URL is used for money transfer processing between users. When registering for a new electronic payment service, registration of a telephone number and password is mandatory. The account ID is issued to the user by the payment server 100, and the user ID is an ID that the user can set at will (or does not have to set). Similarly, the email address and name, address, and date of birth are also information that the user can set at will (or does not have to set). The registration date is the date the user registered for the electronic payment service (the date the account was created). Hereafter, the user instance (electronic payment account) to which this information is associated will be referred to as an account.

[0027] The charge balance is information indicating the balance of electronic money set by the user by sending money to their account in advance. Methods of sending money include sending from an ATM (Automatic Teller Machine) of a designated provider (bank) and sending from a registered bank account. The credit payment setting indicates whether or not the user has completed the settings to enable electronic payments by credit card, and is set to either "Completed" or "Not Completed". The credit payment limit is the monthly limit for credit payments, the credit payment amount is the amount already used for credit payments in the current month, and the available credit payment amount is the amount available for credit payments in the current month, calculated by subtracting the credit payment amount from the credit payment limit. While the diagram shows only one credit payment limit, in reality there are also daily limits, and the lower of these may be set as the credit payment limit. Further details on credit payments will be described later. The payment method setting indicates whether the user will use electronic payment with the charge balance or payment by credit card at that time. The bank account and credit card number information, respectively, refers to the bank account or credit card number (account number, card number) to which funds can be deposited into the electronic payment service. The charge history information is a record of when the user has previously sent money to the electronic payment service to increase the charge balance. The payment history information shows the details of each payment made by the user (date and time, store ID of the store where the purchase was made, payment amount, payment method, etc.).

[0028] Figure 6 shows an example of the contents of the merchant / store information 176. The merchant / store information 176 includes, for example, a first table 176A in which the merchant ID and store ID are associated with the store URL, a second table 176B in which the merchant name and sales amount (as described above) are associated with the merchant ID, and a third table 176C in which the store name is associated with the store ID. In addition to this information, the merchant / store information 176 may also include information such as the merchant or store category, the store's location, and payment patterns.

[0029] The Information Management Unit 140 manages user information 172 and affiliated store / store information 176 based on information obtained from the user terminal device 10 and the second store terminal device 70. The Information Management Unit 140 performs operations such as adding, editing, and deleting new records for user information 172 and affiliated store / store information 176.

[0030] [Electronic payment] When the payment processing unit 130 obtains payment information from the user terminal device 10 or the first store terminal device 50, it refers to the user information 172 to obtain the user's "payment method setting". For users whose "payment method setting" is set to "charge balance", the payment processing unit 130 performs electronic payment as follows: For example, the payment processing unit 130 performs electronic payment by decreasing the charge balance, which is managed in association with the user ID, and increasing the value of the merchant's sales proceeds item. The merchant's sales proceeds item value is not used as electronic money itself, for example, but rather the amount corresponding to the sales proceeds item value is transferred to the bank account in a cycle according to the agreement between the merchant and the electronic payment service.

[0031] The payment processing unit 130 performs electronic payment as follows for users whose "payment method setting" is set to "credit card payment". Credit card payment is a payment method that is carried out in cooperation with a credit card company, which is a separate entity from the operator of the electronic payment service. The operator of the electronic payment service acts as the creditor and allows electronic payment that does not depend on the charge balance within the credit limit. In order to use the credit card payment service, it may be required to obtain a credit card provided by the operator of the electronic payment service. The amount used by credit card payment is settled in a lump sum for the month on the payment date of the following month, for example, by withdrawal from a bank account. In this case, the payment processing unit 130 performs a provisional settlement by adding the settlement amount to the amount used by credit card payment and subtracting the same amount from the available credit card balance. When the closing date arrives, it processes the payment for the current month to be withdrawn on the payment date of the following month as described above, or requests the credit card company operator to perform the said process. If the settlement amount exceeds the available credit card balance at the time of provisional settlement, an error notification is sent back to the payment app 20.

[0032] [Top screen] Figure 7 shows an example of the top screen of the payment app 20. The top screen displays a code image CI. The code image CI includes, for example, barcodes and QR codes. Next to the code image CI is a toggle switch SW for switching between electronic payment using the charged balance or electronic payment using credit. Note that "switch" and "button" are GUIs (Graphical User Interfaces) implemented in cooperation with the touch panel. In Figure 7, "Credit" is displayed, which means that the setting is to perform electronic payment using credit. The user can switch between electronic payment using the charged balance or electronic payment using credit by, for example, swiping the toggle switch SW. The top screen also includes an operation area OA, transition buttons TB1 and TB2. The operation area OA is equipped with buttons that instruct major operations in electronic payment, such as a button to instruct scanning (starting user scanning), a button to send the charged balance to other users, a button to display points earned by the user, and a button to display the history of electronic payments performed by the user. When the transition button TB1 is pressed, the user transitions to a payment screen displaying the code image used for electronic payment and the available balance. When the transition button TB2 is pressed, the user transitions to a screen displaying the available balance for either balance payment or credit payment. In Figure 7, since electronic payment using the charged balance is set, when the transition button TB2 is pressed, the available balance for balance payment is displayed.

[0033] At the bottom of the operating area OA, for example, a group of buttons (switches) M1, M2, ... for launching mini-applications are displayed. A mini-application is an application that operates using the payment application 20 as a platform and provides some kind of service. The service provider develops the mini-application by referring to the SDK (Software Development Kit), which consists of application development programs and technical documents provided by the administrator of the payment application 20. A mini-application is an application that operates when the payment application 20 is running. For example, when the payment application 20 is installed, some or all of the mini-application may be installed, or some or all of the mini-application may be installed from the service server corresponding to the mini-application. For example, when a mini-application is launched, it accesses a service server (not shown) that provides the service corresponding to the mini-application, and the mini-application and the service server cooperate to provide the service to the user. In this case, the service server may be the payment server 100 itself, or it may be an external server different from the payment server 100. In Figure 7, as an example, a button M1 for launching the AI ​​assist app 20A and a button M2 for a mini-app that provides a function to view information about coupons offered by participating stores are displayed. However, buttons for launching various types of mini-apps may be displayed, such as an investment app for managing the charge balance or a payment app for paying public transport fares.

[0034] [AI Assist Function] The AI ​​assist function unit 150 performs natural language processing and dialogue control in response to requests from the AI ​​assist application 20A. The AI ​​assist function unit 150 comprises an orchestrator unit 150A and multiple agent units 150B. The orchestrator unit 150A is responsible for determining the agent unit 150B best suited for the processing based on the input information received from the user terminal device 10 and requesting the processing. Multiple agent units 150B are provided to correspond to each function of the electronic payment service (e.g., help, remittance, charge, etc.), and function as a kind of MCP (Model Context Protocol) server, holding document information 182 that defines the processing it can execute and API information 184 for calling functions that execute these processing as a catalog of tools. Thus, in this embodiment, the agent unit 150B has the functions of both an LLM client that utilizes the LLM server 200 and an MCP server.

[0035] The orchestrator unit 150A is responsible for determining which agent unit 150B (MCP server) is best suited for the processing based on the input information received from the user terminal device 10, and for requesting processing from that agent unit 150B. In addition, the orchestrator unit 150A manages context information (memory), such as the history of past interactions with the user executed on the AI ​​assist application 20A, and uses this context information when determining which agent unit 150B to use. For example, if the user has previously requested a money transfer on the AI ​​assist application 20A, and the current input information from the user is similar to that past transfer request, the orchestrator unit 150A uses this context information to request processing from the agent unit 150B that handles the money transfer. Each agent unit 150B that receives a request executes the processing of its assigned function in accordance with the request from the orchestrator unit 150A.

[0036] Figure 8 shows a scene where a user launches the AI ​​Assist app 20A and enters a request in natural language. The screen shown in Figure 8 is accessed, for example, by the user operating the "AI Assist" button M1 displayed on the top screen of the payment app 20. In other words, the AI ​​Assist app 20A may be implemented as a mini-app developed on the electronic payment service. As shown in Figure 8, the user enters input information IN, for example, "I want to send 3,000 yen to person A," in text or voice. The input information IN may include various contents related to the electronic payment service, such as questions from the user about the electronic payment service or requests to execute functions. The AI ​​Assist app 20A sends this input information IN to the payment server 100.

[0037] When the orchestrator unit 150A of the payment server 100 receives input information IN, it determines which agent unit 150B to request processing from. This determination process is performed using the LLM server 200. The orchestrator unit 150A is also an example of an "acquisition unit," but the acquisition unit, which is a software function unit that receives input information IN, may be provided independently of the orchestrator unit 150A.

[0038] Figure 9 shows an example of an orchestrator prompt 178 that the orchestrator unit 150A inputs to the LLM server 200. The orchestrator prompt 178 includes instruction A1 that instructs the LLM on the role it should play as an orchestrator, information A2 that defines a list of available agents, and a reference A3 to document information 182 that describes the detailed specifications of each agent. In the example in Figure 9, the available agents are defined as a help agent, a remittance agent, a charge agent, and a coupon agent. As shown in Figure 9, information A2 may include a description of the function of each agent in addition to the list of agents. Also, in Figure 9, for the sake of brevity, only the names of the available agents are listed, but in reality, the endpoint (URL) of each agent is also included.

[0039] A help agent is an agent that answers questions from users regarding electronic payment services. The help agent generates answers by referring to document information 182 and web information. A remittance agent is an agent that processes the transfer of electronic money balance from one user to another. A charge agent is an agent that processes the charging (depositing) of electronic money balance for users. A coupon agent is an agent that processes the search, acquisition, and use of coupons available for electronic payment services.

[0040] Here, we will explain in more detail how the LLM server 200 refers to document information 182. In this embodiment, the LLM server 200 improves the accuracy and reliability of the responses it generates by using a technique called Retrieval-Augmented Generation (RAG).

[0041] Specifically, the payment server 100 pre-divides the document information 182 (for example, the help page for the electronic payment service, the functional specifications for each agent, the terms of use, etc.) into predetermined units (chunks). This document information 182 constitutes the knowledge base in this invention. The text information of each chunk is then converted into a numerical vector (embedding) that reflects its semantic content and stored in the vector DB in the storage unit 170.

[0042] When user input information IN or an internal processing request occurs, the LLM server 200 first vectorizes the content of the request. Next, it uses this vector to search the vector database and retrieves one or more chunks that are semantically most similar (i.e., most relevant to the request) as search results. In other words, the LLM server 200 can retrieve the portion of the document that is appropriate to the user's intent as context from the knowledge base. Based on the information retrieved as search results, the LLM server 200 then understands processes such as money transfers and charges. With this configuration, the LLM server 200 can always perform inferences based on accurate and up-to-date document information 182, rather than relying solely on the knowledge inherent in the large-scale language model. This method of vectorizing input information IN and retrieving information can also be executed by the LLM server 200 by specifying it in the orchestrator prompt 178 or the agent prompt 180.

[0043] As described above, the agent unit 150B in this embodiment can be classified into several types according to its role. For example, the agent unit 150B includes a first-type agent unit and a second-type agent unit.

[0044] The first type of agent unit is an agent that generates answers to user questions by referring to document information 182 stored in the memory unit 170. The "Help Agent" shown in Figure 9 corresponds to this. The Help Agent's primary purpose is to provide information, rather than executing APIs to change external states.

[0045] The second type of agent unit is an agent that breaks down the processing requested by the orchestrator unit 150A into multiple tasks and executes those tasks using APIs. The "transfer agent" and "charge agent" shown in Figure 9 are examples of this. These agents call APIs provided by the settlement server 100 to actually change the user's charge balance and perform transfers or charges. In this way, the orchestrator unit 150A appropriately distributes multiple agents with different characteristics, making it possible to respond to a variety of user requests through a single interface.

[0046] The orchestrator unit 150A combines the orchestrator prompt 178 with the user's input information IN ("I want to send 3,000 yen to person A.") and sends it to the LLM server 200. The LLM server 200 infers the user's intent from the input information and determines the most appropriate agent unit 150B. In this embodiment, the LLM server 200 determines the "transfer agent" and returns the result to the orchestrator unit 150A.

[0047] When the orchestrator unit 150A receives a decision result of "remittance agent" from the LLM server 200, it requests the remittance process from the agent unit 150B, which is responsible for the remittance function. Upon receiving the request, the agent unit 150B breaks down the requested process (remittance) into multiple tasks and executes the decomposed tasks. This task decomposition process is also performed using the LLM server 200. In other words, the agent unit 150B, which uses the LLM server 200, is an example of a "task execution unit".

[0048] Figure 10 shows an example of an agent prompt 180 that the agent unit 150B inputs to the LLM server 200. The agent prompt 180 is, for example, a template format that includes variables, and its contents are dynamically set at runtime. This prompt includes instruction A4, which instructs the LLM on the role it should play as a specific agent (in this example, "$agnt"). This prompt functions as a single template, and the variables are dynamically replaced with specific function names such as "transfer" and "charge" depending on the type of agent determined by the orchestrator unit 150A. This eliminates the need to prepare separate prompts for each type of agent, simplifying the overall system management.

[0049] Furthermore, by adopting a template format that includes variables, it will not be necessary to create new prompts when adding new agent units 150B in the future, such as an "invoice payment agent." Simply preparing the specifications for the new function (document information 182 and API information 184) will allow existing prompt templates to be reused to support the new function. This enables rapid and efficient service expansion.

[0050] Instruction A4 states that the requested process should be broken down into tasks, and these tasks should be executed sequentially using the API while interacting with the user using common UI components. The prompts also include a reference A5 to document information 182, which describes the detailed specifications of the function to be handled; a reference A6 to API information 184, which describes the specifications of the available APIs; and a reference A7 to UI component information 186, which describes the specifications of the common UI components described later. Furthermore, rule A8 defines behavior according to the type of agent (whether it is a Type 1 agent or not), preset and confirm the task content from the input information IN, a rule to thoroughly confirm with the user at the end, and referencing user information 172 as needed.

[0051] Here, we will explain a specific example of API information 184. API information 184 is a structured document that defines, for example, the endpoint for calling the API, the HTTP method, the required parameters, and the format of the response. For example, the API information for the remittance function referenced by the "remittance agent" defines that in order to execute the remittance process, the POST HTTP method should be used and a request should be sent to a predetermined endpoint URL. At that time, it is stipulated that the request should include parameters indicating the sender's account ID, the recipient's account ID, and the remittance amount as required parameters, and that it may optionally include a parameter indicating a memo. Furthermore, API information 184 defines that a success notification will be returned as a response if the process is successful, and a failure notification will be returned if an error occurs, such as insufficient balance. For example, some parameters such as the idempotency key are system parameters and are automatically entered. The agent unit 150B collects the information necessary for these requested parameters through interaction (tasks) with the user and executes the function by sending an API request to the settlement processing unit 130, etc., according to the definition in this API information 184. Typically, the agent unit 150B can divide the task into units corresponding to the number of parameters in the API for each process (e.g., a money transfer process).

[0052] The agent unit 150B sends an agent prompt 180 with "$agnt" set to "Transfer" combined with user input information IN to the LLM server 200. Based on this information, the LLM server 200 breaks down the transfer process into multiple tasks, such as "Recipient Input Task," "Transfer Amount Input Task," "Memo Addition Task," and "Final Confirmation Task," determines the UI components to be used in each task, and returns the results to the agent unit 150B.

[0053] The agent unit 150B initiates interaction with the user terminal device 10 based on the task and UI component specifications returned from the LLM server 200. The agent unit 150B generates a UI screen by combining predefined common UI components and sends it to the user terminal device 10. In other words, the agent unit 150B, which uses the LLM server 200, is an example of a "screen generation unit". In this case, the agent unit 150B may not communicate directly with the user terminal device 10 itself, but may instead entrust communication with the user to the orchestrator unit 150A. In that case, the agent unit 150B only needs to pass on the task and UI components to the orchestrator unit 150A.

[0054] [Using common UI components] Figure 11 shows an example of common UI components defined in UI component information 186. These components include a task display box (TSK) that encloses the entire task, an image and name display component (IM), a transition button (BT) used for transitioning to the next task, a numerical input box (NI) for entering numerical values ​​such as amounts, a text box (TB) for entering text, and a search box (SB) for performing keyword searches. The agent unit 150B dynamically generates an interactive screen (server-driven UI) by combining these components. These components can be classified according to their role. For example, the task display box (TSK) that encloses the entire task and the image and name display component (IM) are examples of "task display components". Also, the numerical input box (NI) for entering numerical values ​​such as amounts, the text box (TB) for entering text, and the search box (SB) for performing keyword searches are examples of "task processing components" used by the user to process tasks. Furthermore, the transition button (BT) used for transitioning to the next task is an example of a "task transition component".

[0055] More specifically, for example, UI component information 186 stores the definition information for each UI component in a structured data format (e.g., JSON, XML, or a database table). Each component is defined by several parameters that specify its type, appearance, and behavior. For example, UI component information 186 holds information such as at least the "component ID," "component type," and "parameter set" for each component. The "component ID" is an identifier that uniquely identifies each component. The "component type" is information that indicates which type the component belongs to, as exemplified in Figure 11, such as a task display box (TSK), a transition button (BT), or a numerical input box (NI).

[0056] The "parameter set" includes detailed parameters for controlling the specific display content and layout of a component. For example, a transition button (BT) would include parameters that define visual elements such as "display text" (e.g., "Yes", "Next"), "width", "height", "background color", and "font size". Similarly, an image and name display component (IM) would define parameters such as "image URL", "display name text", and "image size".

[0057] Furthermore, these parameters can also define "actions" that should be performed in response to user actions (e.g., tapping a button). For example, the parameters of a transition button (BT) can define the "action type" (e.g., "task completion notification," "API execution command") and the API endpoint that should be called in conjunction with that action.

[0058] The LLM server 200 refers to this UI component information 186 when breaking down tasks in response to a request from the agent unit 150B. It then dynamically constructs the layout information of the UI screen to be displayed on the user terminal device 10 by selecting the UI components required for each task and specifically setting their parameter sets. The agent unit 150B receives this layout information constructed by the LLM server 200 and transmits it to the user terminal device 10. The AI ​​assist application 20A on the user terminal device 10 receives this layout information and draws the screen accordingly. This configuration makes it possible to provide diverse and flexible interactive screens with only server-side control.

[0059] Furthermore, the large-scale language model installed in the LLM server 200 may be pre-trained (fine-tuned) based on the contents of the UI component information 186 in order to effectively utilize the common UI components used in this embodiment. This training is performed, for example, using training data that links various processes (e.g., money transfer, charging, age verification, coupon acquisition, etc.) and tasks expected within an electronic payment service with candidate UI components (included in the UI component information 186) that are most suitable for executing those processes and tasks. Here, the training data may be data that has been manually mapped in advance by the administrator of the LLM server 200, or it may be the document information 182 itself, or the document information 182 into which processes and tasks and candidate UI components are mapped. By performing such pre-training, the large-scale language model learns which UI components should be selected and combined as candidates from the UI component information 186 for which tasks. As a result, when it receives a task decomposition instruction from the agent unit 150B, the accuracy of generating efficient and appropriate UI screen layout information that is more in line with the user's intent is improved.

[0060] Furthermore, or instead of pre-training, the large-scale language model can dynamically learn the mapping between tasks and UI components from the document information 182. Specifically, when the LLM server 200 receives an instruction from the agent unit 150B to execute a specific task (e.g., "Task to input the transfer amount"), it first searches the vector database using the task name as a query. The document information 182 includes developer documentation that explains the specifications of each task, and includes descriptions such as, "In the transfer process, a numerical input box (NI) and a transition button (BT) are used to present the amount to the user and request confirmation," as well as images representing actual screen examples. Through the RAG mechanism, a chunk containing this description is obtained as a search result and provided to the LLM server 200 as prompt context information.

[0061] By referring to this contextual information, the LLM server 200 can select appropriate UI components based on the document description and generate UI screen layout information, even for unknown tasks not present in the training data. In this way, the LLM server 200 determines its operation self-referentially by using the document describing its own specifications as the learning source. This makes it possible to flexibly extend the system's behavior when adding new tasks or UI patterns by simply updating the document information 182, without having to retrain the large-scale language model.

[0062] [Specific examples of money transfer processing] The following describes a specific example of a money transfer process using the AI ​​assist function, with reference to Figures 12 to 16. Figure 12 is a screen transition diagram showing the input task for the recipient in the money transfer process. First, the agent unit 150B executes the first task, the "recipient input task". As shown in Figure 12, the agent unit 150B generates a UI screen in which a component (USR) that displays the user's image and name, a transition button (BT1), and a search box (SB1) are placed within a task display box (TSK1), and displays it on the user terminal device 10. From the information "Mr. A" included in the user's input information IN, the agent unit 150B searches the user information 172 and displays the information of the most likely candidate "A". If no candidate is found, the agent unit 150B may display only the search box (SB1) along with text information such as "Mr. A was not found". When the user presses the "Yes" button (BT1), the task is considered complete, and the AI ​​assist app 20A sends the details of the process to the agent unit 150B.

[0063] Next, the agent unit 150B executes the "transfer amount input task". Figure 13 is a screen transition diagram showing the transfer amount input task in the transfer process. As shown in Figure 13, the agent unit 150B generates a UI screen in which a numerical input box (NI1) and a transition button (BT2) are placed within a task display box (TSK2). At this time, the information "3000 yen" included in the input information IN is pre-set in the numerical input box (NI1). When the user operates the "Next" button (BT2), the task is completed. When the user operates the "Next" button (BT2), the agent unit 150B may use an API to check whether the user's transfer amount meets the conditions (i.e., whether it is less than or equal to the charge balance) and complete the "transfer amount input task" only if the user's transfer amount meets the conditions.

[0064] Next, the agent unit 150B executes the "add memo task." Figure 14 is a screen transition diagram showing the memo addition task in the remittance process. As shown in Figure 14, the agent unit 150B generates a UI screen in which a user input text box (TB1) and a transition button (BT3) are placed within a task display box (TSK3). The user enters a memo as desired and operates the "Next" button (BT3).

[0065] Next, the agent unit 150B performs a "final confirmation task" before executing the remittance function. This is based on the rules of the agent prompt 180 (A8 in Figure 10). Figure 15 is a screen transition diagram showing the final confirmation task before executing the remittance process. As shown in Figure 15, the agent unit 150B generates a UI screen in the task display box (TSK4) that specifies the recipient and amount, and places a "Yes" button (BT4) prompting final confirmation. When the user operates the "Yes" button (BT4), all tasks are considered complete.

[0066] Once final confirmation is obtained, the agent unit 150B calls the API for executing the remittance based on the API information 184 and requests the settlement processing unit 130 to perform the actual remittance. The settlement processing unit 130 refers to the user information 172 and performs the remittance by subtracting the remittance amount from the user's charge balance and adding the remittance amount to the recipient's charge balance. Figure 16 is a screen transition diagram that notifies the user that the remittance process is complete. Once the processing by the settlement processing unit 130 is complete, the agent unit 150B displays a screen on the user terminal device 10 notifying that the remittance is complete, as shown in Figure 16, and terminates the series of processes.

[0067] Thus, the LLM server 200 breaks down the remittance process into three tasks, excluding the final confirmation task: "Recipient Input Task," "Remittance Amount Input Task," and "Memo Addition Task." This is because the remittance API for executing the remittance process includes three fields as parameters: "Recipient" (e.g., user's account ID), "Remittance Amount," and "Memo." Therefore, the orchestrator prompt 178 may include instructions such as, "When breaking down the tasks, refer to the parameters of the API necessary to execute the $agnt process."

[0068] [Specific examples of charging processes] Next, with reference to Figures 17 to 19, a specific example of the charge process using the AI ​​assist function will be explained. Figure 17 is a screen transition diagram showing the input task for the charge amount in the charge process. As shown in Figure 17, the user inputs input information IN, for example, "I would like to charge 3000 yen," in text or voice. The AI ​​assist application 20A sends this input information IN to the payment server 100. Next, the orchestrator unit 150A sends the received input information IN to the LLM server 200 for interpretation, and as a result selects "Charge Agent".

[0069] Agent unit 150B, which is responsible for the charge agent, breaks down the requested process into two tasks: the "charge amount input task" and the "final confirmation task". It then executes the first task, the "charge amount input task". As shown in Figure 17, agent unit 150B generates a UI screen with a numerical input box (NI2) and a transition button (BT2) placed in the task display box (TSK1), and displays it on the user terminal device 10. At this time, the information "3000 yen" included in the user's input information IN is pre-set in the numerical input box (NI2). When the user operates the "Next" button (BT2), the task is considered complete, and the processing details are sent to agent unit 150B.

[0070] Next, the agent unit 150B performs a "final confirmation task" before executing the charge process. Figure 18 is a screen transition diagram showing the final confirmation task before the charge process is executed. As shown in Figure 18, the agent unit 150B generates a UI screen in the task display box (TSK2) that clearly states the amount to be charged and includes a "Yes" button (BT4) prompting final confirmation. When the user operates the "Yes" button (BT4), all tasks are considered complete.

[0071] Once final confirmation is obtained, the agent unit 150B calls the API for charge execution based on the API information 184 and requests the payment processing unit 130 to perform the actual charge processing. Figure 19 is a screen transition diagram that notifies the user that the charge processing is complete. When the processing by the payment processing unit 130 is completed, the agent unit 150B displays a screen on the user terminal device 10 notifying that the charge is complete, as shown in Figure 19, and terminates the series of processes.

[0072] [Error handling] Although the above describes an example where a task is completed successfully, the agent unit 150B can also be configured to handle errors during task execution. More specifically, for example, the agent prompt 180 can be configured to "notify the orchestrator unit 150A with detailed error information if an API execution error occurs" and "restart the task when the orchestrator unit 150A notifies it that the error has been resolved," and the orchestrator prompt 178 can be configured to "start the error resolution process when the agent unit 150B notifies it of an error" and "notify the agent unit 150B that the error has been resolved."

[0073] Figure 20 shows an example of the processing flow when an error occurs. Figure 20 illustrates an example where a user attempts to send 3,000 yen even though they only have a charge balance of 2,000 yen. In this case, for example, an error occurs in the "Inputting the transfer amount task" because the transfer amount exceeds the charge balance, and the agent unit 150B takes over processing to the orchestrator unit 150A. As a result, the orchestrator unit 150A decides to start the process to resolve the error, i.e., to start the charge process, and requests the agent unit 150B, which is the charge agent, to perform the process.

[0074] Once the charge process is complete, the orchestrator unit 150A returns the process to the agent unit 150B, which is the remittance agent. The agent unit 150B then completes the remittance process after going through the "add memo task" and the "final confirmation task for the charge process." In Figure 20, an error occurs in the "enter remittance amount task" as an example, but it is also possible that an error may occur at the time when the remittance process is finally executed after going through the "final confirmation task for the remittance process." In that case, the process transitions from the "final confirmation task for the remittance process" to the charge process, and after the charge is completed, it returns to the "final confirmation task for the remittance process" again to complete the remittance process. In this way, by appropriately configuring the orchestrator prompt 178 and the agent prompt 180, even if an error occurs during the processing by one agent unit 150B, the process can be taken over to another agent unit 150B and the original intended process can be completed.

[0075] Figure 20 illustrates an example where the charge agent takes over processing in response to an error that occurs while the remittance agent is executing the process. However, the present invention is not limited to such a configuration. Depending on the user's input information IN and demands, the orchestrator unit 150A may coordinate multiple processes by multiple agent units to execute the processes of each agent unit in succession. For example, if a user inputs "I want to charge 3000 yen and then send 1000 yen to person A" as input information IN, the orchestrator unit 150A may first request processing from the charge agent, and then request processing from the remittance agent. In that case, the charge agent will first execute the tasks shown in Figures 17 and 18, and then the remittance agent will execute the tasks shown in Figures 12 to 15. In this way, even if a user inputs complex input information IN that includes multiple intentions, the orchestrator unit 150A's coordination allows the user to achieve their objective by processing the tasks as instructed.

[0076] [Summary of processing flow] Figure 21 is a sequence diagram showing an example of the processing flow of the AI ​​assist function. First, when the user inputs information into the AI ​​assist application 20A (S11), the AI ​​assist application 20A sends the input information to the orchestrator unit 150A of the payment server (S12). The orchestrator unit 150A sends the input information and the orchestrator prompt 178 to the LLM server 200 (S13) and requests the server to determine the agent unit 150B to which the request will be made. The LLM server 200 determines the agent unit 150B (S14) and notifies the orchestrator unit 150A (S15).

[0077] The orchestrator unit 150A passes the input information to the determined agent unit 150B and requests processing (S16). The agent unit 150B sends the input information and agent prompts to the LLM server 200 (S17) and requests that the processing be broken down into tasks and the UI components for each task be determined. The LLM server 200 performs the task breakdown and UI component determination (S18) and notifies the agent unit 150B (S19).

[0078] The agent unit 150B transmits information about the determined task and UI components to the AI ​​assist application 20A via the orchestrator unit 150A, and displays the screen (S20). When the user operates the screen and processes the task (S21), the processing details are transmitted from the AI ​​assist application 20A to the agent unit 150B (S22). The agent unit 150B relays the task processing details to the LLM server 200, and the LLM server 200, depending on the task processing details, completes the processing by coordinating with other internal functions (such as the payment processing unit 130) using an API, for example (S23), and notifies the agent unit 150B of the completion of the processing (S25). The agent unit 150B then notifies the AI ​​assist application 20A of the completion of the processing (S26), and the series of processes ends.

[0079] Note that in the sequence diagram in Figure 21, the processes from S19 to S24 are shown as a single process (i.e., a single task process). However, if there are multiple processing tasks, the processes from S19 to S24 will loop.

[0080] According to the embodiment described above, based on the input information entered by the user of the electronic payment service, the orchestrator unit determines an appropriate agent unit from among multiple agent units and requests processing. The determined agent unit breaks down the requested processing into multiple tasks and executes the tasks while interacting with the user through a UI screen generated using common UI components. As a result, the user can use various functions of the electronic payment service, such as sending money and charging, simply by entering requests in natural language, without having to perform complex operations. In other words, according to this embodiment, the AI-assisted functions in the electronic payment service can be improved for the user. As a result, user engagement with the electronic payment service can be improved.

[0081] Furthermore, according to this embodiment, since task flows and UI screens to respond to user requests are dynamically generated on the LLM server 200 side, new functions (new agent units) can be added or existing functions (existing agent units) can be flexibly modified simply by preparing the specifications for the new functions (document information 182 and API information 184) without updating the AI ​​assist application 20A, which is the client application. This increases the development efficiency and scalability of the AI ​​assist service. In other words, according to this embodiment, developers can improve the AI ​​support functions in electronic payment services.

[0082] [Image generation function] As mentioned above, even if a user who wishes to send money using an electronic payment service is unfamiliar with its specifications and procedures, they can proceed with the transfer by simply entering their request in natural language into the AI-assisted app 20A. This eliminates the need for users to research the transfer function and improves their user experience. In addition, if value can be added to the customer experience when using the transfer function, it may be possible to increase user engagement with the electronic payment service. For example, one way to add value to the customer experience is to facilitate communication between users during the transfer process.

[0083] Therefore, the payment server 100 according to this embodiment provides a function to send an image generated using the LLM server 200 to the recipient user when a remittance is executed. This aims to provide a value-added customer experience that goes beyond mere money transfer and to improve user engagement with the electronic payment service. The image generation function will be described in detail below, separately for the case of remittance using the AI ​​assist function and the case of using the normal remittance function. In the following description, the AI ​​assist function unit 150 (particularly the orchestrator unit 150A and agent unit 150B) is an example of an "information processing device" and realizes the functions of the "acquisition unit," "generation unit," "display control unit," "transmission unit," and "proposal unit" in the claims.

[0084] [When using the AI-assisted remittance function] First, with reference to Figures 22 and 23, we will explain the case where an image generation function is incorporated into a remittance process using the AI-assisted function. Figures 22 and 23 are diagrams illustrating an example of screen transitions involving image generation in a remittance process using the AI-assisted function. More specifically, Figure 22 shows the image generation task in the remittance process, and Figure 23 shows the final confirmation task after image generation and the remittance completion screen.

[0085] First, as explained with reference to Figure 8, when a user inputs information IN (an example of a "transfer request") such as "I want to send 3,000 yen to person A," the orchestrator unit 150A acquires this information and requests processing from the agent unit 150B, which is the transfer agent. The transfer agent works in cooperation with the LLM server 200 to break down the transfer process into multiple tasks. In this embodiment, the transfer agent generates, for example, an "image generation task (TSK3')" in addition to the "recipient input task (TSK1)" (Figure 12), the "transfer amount input task (TSK2)" (Figure 13), and the "memo addition task (TSK3)" (Figure 14). This image generation task TSK3' is executed, for example, between the memo addition task TSK3 and the final confirmation task TSK4 (Figure 15). Note that the memo addition task and the image generation task may be integrated into a single task and executed on the same screen.

[0086] The left side of Figure 22 shows an example of the initial screen for the image generation task (TSK3'). The remittance agent uses common UI components (see Figure 11) to generate a UI screen within the task display box TSK3', which contains a text box TB2 for entering text to generate the image, a "Generate Image" button BT5 (an example of an "option") to instruct image generation, and a "Next" button BT6 to proceed to the next task. This screen is displayed on the user terminal device 10 of the first user (an example of a "first user terminal device"). This screen allows the user to select whether or not to generate an image to send to the user terminal device of the other user (an example of a "second user terminal device"), who is the recipient of the remittance.

[0087] When a user wants to generate an image, they enter text (a prompt) in text box TB2 describing the content of the image they want to generate. In the example in Figure 22, the text entered is "An illustration of a man and a woman eating delicious ramen." When the user operates button BT5, the remittance agent sends the entered text to the LLM server 200.

[0088] The LLM server 200 generates an image based on the received text and returns it to the remittance agent. Here, the LLM server 200 is equipped with an image generation model (e.g., a diffusion model) that has been trained to generate images from text (text-to-image). Note that the image generation model equipped in the LLM server 200 may be a different model from the large-scale language model that performs text generation as described in Figures 9 and 10, or it may be a multimodal large-scale language model that can handle both text and images.

[0089] When the remittance agent receives an image from the LLM server 200, it displays the generated image IM in the task display box TSK3', as shown on the right side of Figure 22. The user can check the generated image IM and, if it does not meet their expectations, can regenerate the image by operating button BT5 again. A button (not shown) for deleting the generated image IM may also be displayed. When the user operates button BT6, the image generation task TSK3' is completed and the process transitions to the next final confirmation task TSK4. If the user operates button BT6 without generating an image, the image will not be sent.

[0090] Figure 23 shows the final confirmation task TSK4 and the screen after the transfer is completed. As shown on the left side of Figure 23, the transfer agent displays the transfer details (recipient, amount), the memo entered in Figure 14 (for example, "Let's go for ramen again!"), and the image IM generated in Figure 22 within the task display box TSK4. When the user approves the transfer by operating button BT4, the transfer agent requests the settlement processing unit 130 to process the transfer and sends the transfer notification, memo, and image IM to the recipient user's terminal device.

[0091] As shown in Figure 23 (right), the user terminal device 10 of the sending user displays a screen indicating that the transfer has been completed. This screen also displays, for example, the sent memo and image IM. The receiving user can also view the image IM and memo sent from the sender on their own user terminal device.

[0092] [When using the standard money transfer function] Next, referring to Figures 24 to 26, we will explain that the image generation function can be used in a normal remittance flow where the user manually performs the remittance operation without using the AI ​​assistance function. Figures 24 to 26 show examples of screen transitions involving image generation in a normal remittance process.

[0093] First, as the first step in the remittance process, the left side of Figure 24 is the recipient selection screen, which is accessed when, for example, the "Send" button on the top screen of the payment app 20 shown in Figure 7 is pressed. The user selects the recipient (for example, user A) from their history, friends list, or group list.

[0094] Once the recipient is selected, the system transitions to the screen shown on the right side of Figure 24 as the second transfer step, where the user selects either the transfer button BT7 (an example of a "transfer request") or the request button BT8. The "request" button is a function that requests payment from the recipient. The image generation function in this embodiment can be used not only when transferring money but also when requesting payment. For example, when a user makes a request, they can attach an AI-generated image and send it to the recipient, thereby facilitating smoother communication.

[0095] When the user presses the "Send" button BT7, the system transitions to the amount input screen shown on the left side of Figure 25 as the transfer step 3. The user enters the transfer amount (for example, 1000 yen) and presses the "Next" button BT9.

[0096] When button BT9 is pressed, the system transitions to the memo addition and image generation screen, shown on the right side of Figure 25, as the remittance step 4. On this screen, the remittance agent sends and displays information to the user terminal device 10, including a text box TB1 for entering a memo, a text box TB2 for entering text to generate an image, a button BT10 to instruct image generation, and a button BT11 to proceed to the next step. In the normal remittance flow, a screen is also provided for the user to choose whether or not to generate an image.

[0097] When the user enters text (for example, "an illustration of a man and a woman eating delicious ramen") into text box TB2 and operates button BT10, the payment app 20 sends the entered text to the payment server 100. The remittance agent works with the LLM server 200 to generate an image and returns it to the payment app 20. The payment app 20 displays the generated image (not shown) on the screen. When the user operates the "Next" button BT11, the process moves to the next step.

[0098] As step 5 of the remittance process, the left diagram in Figure 26 shows the final confirmation screen. This screen displays the remittance details (recipient, amount), as well as the entered memo and the generated image. When the user approves the remittance by pressing the "Yes" button BT12, the payment application 20 requests the payment server 100 to execute the remittance. The payment server 100 executes the remittance process and also sends the memo and image to the user terminal device of the recipient.

[0099] As step 6 of the remittance process, the right-hand image in Figure 26 shows an example of a message screen (chat screen) displayed after the remittance is complete. This screen displays the remitted amount, a memo, and a generated image in chronological order. The recipient can then review this information. In this way, incorporating an image generation function into the normal remittance flow can facilitate communication between users. In other words, this can improve user engagement with electronic payment services.

[0100] [Other features] (AI-generated memos) In the above embodiment, a case was described in which the user manually enters a memo, but the memo may also be generated by AI. For example, in the remittance step 4 of Figure 25, a button (not shown) for generating a memo using AI may be provided. When the user operates this button, the remittance agent uses the LLM server 200 to generate memo candidates appropriate to the context of the remittance. For example, based on the recipient, the amount, or the settlement history information described later, it can generate an appropriate message (e.g., "This is for yesterday's meal. Thank you!") and suggest (preset) it in the text box TB1.

[0101] (Automatic generation and suggestion of prompts) In the above embodiment, an example was described in which the user manually inputs a prompt (TB2) for image generation, but AI may assist in inputting the prompt. For example, the remittance agent may use the LLM server 200 to generate appropriate prompt candidates based on the content of the memo entered by the user and propose them in the text box TB2. For example, if the memo is "Let's go for ramen again!", a prompt such as "Illustration of eating ramen" may be automatically generated.

[0102] Furthermore, the remittance agent may use the user's past behavioral history (e.g., payment history information) when generating prompts. For example, if the user made a payment at a specific merchant (e.g., a pizza restaurant) immediately before making a remittance, the remittance agent can refer to the payment history information in user information 172 and automatically generate a prompt related to that payment (e.g., "an illustration of someone happily eating pizza") and suggest it to the user. This allows the user to easily generate an image appropriate to the remittance situation without having to think of a prompt themselves.

[0103] (Use and editing of photographs) In the above embodiment, an example of generating an image from scratch using AI was described, but a photograph taken by the user may also be used. For example, the image generation screen (Figure 22 and Figure 25 right) may be provided with a function for the user to upload a photograph stored on their user terminal device 10 (for example, a photograph taken during a meal). The remittance agent may then send the uploaded photograph to the LLM server 200 and process it to look like an illustration or painting. This allows the user to send a more personalized image to the recipient, further improving the user's remittance experience.

[0104] [summary] As described above, the payment server 100 according to this embodiment acquires a remittance request to a second user entered by a first user of the electronic payment service. Upon acquiring the remittance request, the server displays options (e.g., buttons BT5 and BT10) on the first user's terminal device 10 for selecting whether or not to generate an image to send to the second user's terminal device. Then, based on the first user's selection, the LLM server 200 generates an image and sends the generated image to the second user's terminal device. This allows the remittance function in the electronic payment service to enrich communication between users and provide added value as a customer experience by adding an AI-generated image, in addition to simply transferring money. As a result, user engagement with the electronic payment service can be improved.

[0105] Furthermore, in this embodiment, when generating an image, the image is generated based on the text entered by the first user. This allows the user to easily generate and send an original image that aligns with their intentions and the context of the remittance. In addition, in this embodiment, a memo associated with the remittance request is obtained, and based on the content of the memo, text (prompt) for generating the image is generated and proposed to the user terminal device 10. This reduces the effort required for the user to enter text for image generation.

[0106] Furthermore, in this embodiment, text for generating an image is generated based on the first user's payment history and proposed to the user terminal device 10. For example, by making it easier to generate an image related to the most recent payment, it becomes possible to easily send an image that matches the purpose of the remittance. In addition, in this embodiment, the original image uploaded by the first user is obtained, and the original image is processed to generate the image to be sent. This enables more personalized communication based on memories such as photographs held by the user.

[0107] Furthermore, in this embodiment, an image generation function is provided in the AI ​​assist function (AI assist app 20A) that performs processing based on natural language input information IN. In particular, the AI ​​assist function decomposes the remittance process into multiple tasks and executes an image generation task (TSK3') as one of those tasks. This makes it possible to seamlessly integrate the image generation function into the interactive remittance flow.

[0108] Although embodiments for carrying out the present invention have been described above using examples, the present invention is not limited in any way to these embodiments, and various modifications and substitutions can be made without departing from the spirit of the present invention. [Explanation of symbols]

[0109] 10. User terminal device 20 Payment Apps 20A AI Assist App 100 Payment Servers 150 AI Assist Function Unit 150A Orchestrator Section 150B Agent Department 200 LLM servers

Claims

1. An acquisition unit that acquires a remittance request to a second user entered by the first user of an electronic payment service, In response to receiving the aforementioned remittance request, the display control unit displays an option on the first user terminal device of the first user for selecting whether or not to generate an image to be sent to the second user terminal device of the second user, A generation unit that, in response to the first user selecting the option, acquires the text entered by the first user and generates an image to be transmitted to the second user terminal device using an image generation model based on the text, A settlement processing unit that performs a remittance process based on the aforementioned remittance request, which subtracts the electronic money of the first user and adds the electronic money of the second user, The system includes a transmission unit that transmits the image generated in connection with the execution of the aforementioned remittance process to the second user terminal device. Information processing device.

2. The acquisition unit acquires the memo associated with the remittance request, The system further includes a proposal unit that generates candidate texts for generating the image by inputting the contents of the memo into a large-scale language model and proposes them to the first user terminal device. The information processing apparatus according to claim 1.

3. The system further includes a proposal unit that generates candidate texts for generating the image by inputting the payment history of the first user into a large-scale language model and proposes them to the first user terminal device. The information processing apparatus according to claim 1.

4. The generation unit acquires the original image uploaded by the first user, inputs the text and the original image into the image generation model, and generates the image by processing the original image. The information processing apparatus according to claim 1.

5. The acquisition unit acquires the remittance request by inputting the input information entered in natural language by the first user into a large-scale language model and analyzing the intent of the input information. The information processing apparatus according to claim 1.

6. The system further includes a task decomposition unit that decomposes the remittance process based on the input information into multiple tasks by inputting the aforementioned input information into a large-scale language model. The display control unit, in an image generation task which is one of the plurality of tasks for selecting whether or not to generate the image, displays the options. The information processing apparatus according to claim 5.

7. The generation unit generates a memo to be sent to the second user terminal device by inputting the context of the remittance request into a large-scale language model. The transmitting unit transmits the generated image and the generated memo to the second user terminal device. The information processing apparatus according to claim 1.

8. Computers The system retrieves a transfer request to a second user, entered by the first user of the electronic payment service. Upon receiving the aforementioned remittance request, the first user's terminal device displays an option to select whether or not to generate an image to be sent to the second user's terminal device. In response to the first user selecting the option, the system retrieves the text entered by the first user, and uses an image generation model based on the text to generate an image to be transmitted to the second user terminal device. Based on the aforementioned remittance request, a remittance process is executed that subtracts the electronic money of the first user and adds the electronic money of the second user. The image generated upon execution of the aforementioned remittance process is transmitted to the second user terminal device. Information processing methods.

9. On the computer, The system retrieves a remittance request from the first user of an electronic payment service to the second user. Upon receiving the aforementioned remittance request, the first user's terminal device is instructed to display an option to select whether or not to generate an image to be sent to the second user's terminal device. In response to the first user selecting the option, the text entered by the first user is acquired, and an image is generated to be transmitted to the second user terminal device using an image generation model based on the text. Based on the aforementioned remittance request, the remittance process is executed, which subtracts the electronic money of the first user and adds the electronic money of the second user. The image generated upon execution of the aforementioned remittance process is transmitted to the second user terminal device. program.

10. An acquisition unit that acquires a remittance request to a second user and a memo associated with the remittance request, entered by the first user of the electronic payment service. In response to receiving the aforementioned remittance request, the display control unit displays an option on the first user terminal device of the first user for selecting whether or not to generate an image to be sent to the second user terminal device of the second user, A generation unit generates a prompt for generating an image by inputting the contents of the memo into a large-scale language model, and generates an image by inputting the prompt into an image generation model in response to the first user selecting the option. A settlement processing unit that performs a remittance process based on the aforementioned remittance request, which subtracts the electronic money of the first user and adds the electronic money of the second user, The system includes a transmission unit that transmits the image and memo generated in connection with the execution of the aforementioned remittance process to the second user terminal device. Information processing device.

11. Computers The system retrieves a remittance request to a second user and a memo associated with the remittance request, entered by the first user of the electronic payment service. Upon receiving the aforementioned remittance request, the first user's terminal device displays an option to select whether or not to generate an image to be sent to the second user's terminal device. By inputting the contents of the aforementioned memo into a large-scale language model, a prompt for generating an image is generated, and in response to the first user selecting the option, the prompt is input into an image generation model to generate an image. Based on the aforementioned remittance request, a remittance process is executed that subtracts the electronic money of the first user and adds the electronic money of the second user. The generated image and memo are transmitted to the second user terminal device upon execution of the aforementioned remittance process. Information processing methods.

12. On the computer, The system retrieves a remittance request to a second user and a memo associated with the remittance request, entered by the first user of the electronic payment service. Upon receiving the aforementioned remittance request, the first user's terminal device is instructed to display an option to select whether or not to generate an image to be sent to the second user's terminal device. By inputting the contents of the aforementioned memo into a large-scale language model, a prompt for generating an image is generated, and in response to the first user selecting the option, the prompt is input into an image generation model to generate an image. Based on the aforementioned remittance request, the remittance process is executed, which subtracts the electronic money of the first user and adds the electronic money of the second user. The image and memo generated upon execution of the aforementioned remittance process are transmitted to the second user terminal device. program.

Images