Information processing device, information processing method, program, and information processing system

Abstract

To improve the voice-based operability of an application providing an electronic settlement function.SOLUTION: An information processing device controls the operation of an application program that operates on a user's terminal device based on voice information of the user that is input to the terminal device, and includes: an acquisition unit configured to acquire text data of voice information that is input to the terminal device, and indicates the content of an utterance related to what the user wants to achieve using the application program; and a function specification unit configured to input the text data based on learning results obtained by learning a relation between various functions of the application program and items that can be achieved by the various functions and specify a function to cause the application program to execute it in order to achieve what the user wants to achieve.SELECTED DRAWING: Figure 7

Description

[Technical field]

[0001] The present invention relates to an information processing device, an information processing method, a program, and an information processing system. [Background technology]

[0002] 2. Description of the Related Art Conventionally, there has been proposed a payment device that uses a voice reading function to assist a user in making an electronic payment (eg, Patent Document 1). [Prior art documents] [Patent documents]

[0003] [Patent Document 1] JP 2015-060281 A Summary of the Invention [Problem to be solved by the invention]

[0004] Conventional technology could help users make electronic payments by reading out the price of a product aloud, but the range of actions it could support was limited and it was not necessarily very convenient.

[0005] The present invention has been made in consideration of the above circumstances, and one of its objectives is to provide an information processing device, an information processing method, a program, and an information processing system that can improve the voice operability of an application that provides electronic payment functions. [Means for solving the problem]

[0006] One aspect of the present invention is an information processing device that controls the operation of an application program running on a user's terminal device based on user voice information input to the terminal device, and includes an acquisition unit that acquires text data of the voice information input to the terminal device, the voice information indicating the content of the speech related to what the user wants to achieve using the application program, and a function identification unit that inputs the text data and identifies functions to be executed by the application program to achieve what the user wants to achieve, based on learning results that have learned the relationship between various functions of the application program and matters that can be achieved by the various functions. Effect of the Invention

[0007] According to one aspect of the present invention, it is possible to provide an information processing device, an information processing method, a program, and an information processing system that can improve the voice operability of an application that provides an electronic payment function. [Brief description of the drawings]

[0008] [Figure 1] FIG. 1 is a diagram showing an example of a configuration for realizing an electronic payment service. [Diagram 2] This is a sequence diagram (part 1) illustrating the general flow of electronic payment. [Diagram 3] This is a sequence diagram (part 2) illustrating the general flow of electronic payment. [Figure 4] 1 is a configuration diagram of a payment server 100 according to a first embodiment. [Diagram 5] FIG. 13 is a diagram showing an example of the contents of user information 172. [Figure 6] FIG. 13 is a diagram showing an example of the contents of affiliated store / store information 176. [Figure 7] FIG. 2 is a diagram illustrating an example of a functional configuration of a system that realizes a voice operation function of payment application 20. [Figure 8]13 is a sequence chart showing an example of a process flow in which a payment application 20, a conversion server 200, and a payment server 100 cooperate with each other to realize a voice operation function. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0009] Hereinafter, with reference to the drawings, an embodiment of an information processing device, an information processing method, a program, and an information processing system of the present invention will be described. Various devices for providing services to users and performing internal analysis, such as a "server", a "management device", and an "information providing device" that will be described below, may be realized by a group of distributed devices, and each device may be operated by a different business operator. Furthermore, the owner of the hardware of the device (the provider of the cloud server) and the business that actually operates the device may also be different. An application program and a payment server work together to provide an electronic payment service. In the following description, the application program is referred to as a payment application. An electronic payment service is a service that supports payments related to the purchase of goods and services at a store. A store is, for example, a physical store (real store) that exists in the real world, but may also include a virtual store for electronic commerce. A virtual store may include a store provided by an entity different from the operator of the electronic payment service. In that case, when making a payment for shopping at a virtual store, the screen is controlled to transition to an interface screen of the electronic payment service. In an electronic payment service, a store is treated as belonging to, for example, an affiliated store (brand), and processing such as payment when a purchase is made at a store is mainly performed between the user and the affiliated store. Alternatively, processing such as payment may be carried out between the user and the store.

[0010] Furthermore, the payment application and the conversion server work together to provide a voice-based operation interface for the payment application. The payment application inputs the user's voice and transmits the voice data to the conversion server. The conversion server recognizes what the user wants to achieve by using the payment application based on the user's voice data, determines a command for operating the payment application so as to achieve the desired result, generates command information for causing the payment application to execute the command, and transmits it to the payment application. The payment application operates based on the command information received from the conversion server and cooperates with the payment server, thereby achieving what the user wants to achieve. In the following description, the conversion server is configured as a device separate from the payment server, but the payment server may have the function of the conversion server. The payment application may also have the function of the conversion server.

[0011] [Electronic payment service] Fig. 1 is a diagram showing an example of a configuration for realizing an electronic payment service. The electronic payment service is realized mainly by a payment server 100. The payment server 100 communicates with, for example, one or more user terminal devices 10, one or more first store terminal devices 50, and one or more second store terminal devices 70 via a network NW. The network NW includes, for example, the Internet, a LAN (Local Area Network), a wireless base station, a provider device, and the like.

[0012] The user terminal device 10 is, for example, a portable terminal device such as a smartphone or a tablet terminal. The user terminal device 10 is a computer device having at least an optical reading function, a communication function, a display function, an input reception function, and a program execution function. In the following description, components for realizing these functions are referred to as a camera, a communication device, a touch panel, a CPU (Central Processing Unit), etc. In the user terminal device 10, a processor such as a CPU executes a payment application 20, thereby operating to provide an electronic payment service to a user in cooperation with a payment server 100. The payment application 20 is installed in the user terminal device 10 from, for example, an application store, and controls the camera, communication device, touch panel, etc.

[0013] In addition, the user terminal device 10 communicates with the conversion server 200 via the network NW. The payment application 20 provides the user with a voice-based operation support function for the payment application 20 (hereinafter referred to as the "voice operation function") in cooperation with the conversion server 200. The voice operation function will be described in detail later.

[0014] The first store terminal device 50 is installed, for example, in a store. The first store terminal device 50 is a computer device having at least a product price acquisition function, an optical reading function, a program execution function, and a communication function. The first store terminal device 50 includes a so-called POS (Point of Sale) device, and the product price acquisition function and the optical reading function may be realized by the POS device. The store code image 60 is placed in the store, and is a code image such as a QR code (registered trademark) printed on a paper or plastic medium. The store code image 60 may be displayed on a display placed in the store (which may be the display of a terminal device such as a smartphone).

[0015] The second store terminal device 70 is used by the operator of the affiliated store. The second store terminal device 70 is a smartphone, a tablet terminal, a personal computer, or the like. An interface 72 for affiliated stores runs on the second store terminal device 70. The interface 72 for affiliated stores may be an app for affiliated stores or a browser. The interface 72 for affiliated stores accepts coupon settings and the like made by the operator of the affiliated store and transmits them to the payment server 100. The second store terminal device 70, which is a smartphone, has the function of displaying a code image corresponding to a store code image and reading a code image displayed by the user terminal device 10 by executing the app for affiliated stores.

[0016] The payment server 100 realizes electronic payment based on payment information received from the user terminal device 10 or the first store terminal device 50. The first store terminal device 50 may include a POS device and an affiliated store server, in which case the payment information is sent from the POS device via the affiliated store server to the payment server 100. In the following explanation, no distinction is made between these two and it is assumed that the payment information is sent from the first store terminal device 50.

[0017] 2 and 3 are sequence diagrams illustrating the general flow of electronic payment. There may be two patterns of electronic payment: pattern 1 and pattern 2.

[0018] In the case of pattern 1 (hereinafter referred to as user scan) shown in FIG. 2, the user terminal device 10 with the payment application 20 activated decodes the store code image 60 by the optical reading function (S1). The store code image 60 includes store URL (Uniform Resource Locator) information. This store URL is an electronic payment service domain to which store-identifying information is added, and is associated with an affiliated store ID, a store ID, etc. in the payment server 100 (described later). The payment application 20 transmits the first payment information including the store URL and the account ID to the payment server 100 (S2). The payment server 100 searches for store information (described later) from the affiliated store ID and the store ID corresponding to the store URL, acquires the affiliated store name and the store name information (S3), and transmits it to the payment application 20 (S4). The user inputs the payment amount into the user terminal device 10 on the screen on which the affiliated store name and the store name are displayed (S5). Then, the user terminal device 10 generates second payment information including at least the payment amount, and transmits it to the payment server 100 (S6). The payment server 100 performs electronic payment based on the received second payment information (S7). The payment server 100 then transmits a payment completion notice (information for displaying a payment completion screen) to the payment application 20 (S8), and the payment application 20 displays the payment completion screen (S9). Note that when the store code image 60 is displayed on a display installed in the store, the store code image 60 may include not only the store URL but also information on the payment amount. In this case, the procedure in which the user inputs the payment amount is omitted, and the information on the payment amount is included in the first payment information and transmitted to the payment server 100. Information on the affiliated store name and the store name may be included in the payment completion screen and displayed.

[0019] In the case of pattern 2 (hereinafter referred to as store scan) shown in FIG. 3, when the payment application 20 is started, when a payment operation is performed in the payment application 20, when an automatic update timing (e.g., every minute) occurs, and at other timings, the payment application 20 transmits a request for issuing a one-time code to the payment server 100 (S11). The payment server 100 generates a one-time code (S12) and transmits it to the payment application 20 (S13). The payment application 20 displays a code image such as a QR code or a barcode generated based on the one-time code (S14). The user holds (presents) the display surface of the user terminal device 10 over the first store terminal device 50, and the first store terminal device 50 decodes the code image by an optical reading function and obtains the one-time code, etc. (S15). The first store terminal device 50 then generates payment information including the one-time code, payment amount, affiliated store ID, store ID, etc., and transmits it to the payment server 100 (S16). The payment amount information is acquired in advance by reading a barcode or manually entering it. The payment server 100 identifies the user corresponding to the one-time code based on the received information and performs electronic payment (S17). The payment server 100 then transmits a payment completion notice to the payment application 20 (S18), and the payment application 20 displays a payment completion screen (S19).

[0020] Note that electronic payment may be performed using only one of the above patterns. Furthermore, the "account ID" described in FIG. 2 may be other information (e.g., a phone number) that can be used as user identification information. Furthermore, issuance of a one-time code may be omitted in the store scan, and the payment application 20 may display a code image generated based on the user's account ID. In this case, the payment server 100 identifies the user corresponding to the account ID instead of identifying the user corresponding to the one-time code.

[0021] [Payment server] FIG. 4 is a configuration diagram of the payment server 100 according to the first embodiment. The payment server 100 includes, for example, a communication unit 110, a payment content providing unit 120, a payment processing unit 130, an information management unit 140, and a storage unit 170. The components other than the communication unit 110 and the storage unit 170 are realized by, for example, a hardware processor such as a CPU executing a program (software). Some or all of these components may be realized by hardware (including circuitry) such as an LSI (Large Scale Integration), an ASIC (Application Specific Integrated Circuit), an FPGA (Field-Programmable Gate Array), or a GPU (Graphics Processing Unit), or may be realized by cooperation between software and hardware. The program may be stored in advance in a storage device (storage device having a non-transient storage medium) such as an HDD (Hard Disk Drive) or a flash memory, or may be stored in a removable storage medium (non-transient storage medium) such as a DVD or a CD-ROM, and may be installed in the storage device by mounting the storage medium in a drive device.

[0022] The storage unit 170 is a HDD, a flash memory, a RAM (Random Access Memory), etc. The storage unit 170 may be a NAS (Network Attached Storage) device that the payment server 100 can access via a network. The storage unit 170 stores information such as user information 172, payment content information 174, and affiliated store / shop information 176.

[0023] The communication unit 110 is a communication interface for connecting to the network NW. The communication unit 110 is, for example, a network interface card.

[0024] The payment content providing unit 120 has, for example, a function of a Web server, and provides information (content) for displaying various screens of the electronic payment service to the user terminal device 10. The payment content providing unit 120 appropriately reads necessary content from the payment content information 174 and provides it to the user terminal device 10. The user terminal device 10 accepts various inputs by the user while the content is being played by the payment application 20, and transmits the above-mentioned payment information and the like to the payment server 100.

[0025] The payment processing unit 130 performs payment processing based on the payment information transmitted by the user terminal device 10 or the first store terminal device 50. The payment processing unit 130 performs payment processing while referring to the user information 172.

[0026] FIG. 5 is a diagram showing an example of the contents of the user information 172. The user information 172 is an example of the registration information of a user. The user information 172 is, for example, a user URL, an account ID, a telephone number, a password, as well as information associated with an email address, a user ID, a name, an address, a date of birth, a registration date, a charge balance, a post-payment setting, a post-payment limit, a post-payment usage amount, a post-payment available amount, a payment method setting, a bank account, a credit card number, a charge history information, and a payment history information. The user URL is used for a remittance process between users. When registering for the electronic payment service, it is necessary to register a telephone number and a password. The account ID is issued to the user by the payment server 100, and the user ID is an ID that can be set by the user at will (does not have to be set). Similarly, the email address, and the name, address, and date of birth are information that can be set by the user at will (does not have to be set). The registration date is the date on which the user registered for the electronic payment service (the date on which the account was created). Hereinafter, the user's instance (electronic payment account) to which this information is associated will be referred to as an account.

[0027] The charge balance is information indicating the balance of electronic money that is set by a user by transferring money to an account in advance. The means of transfer include transfer from an ATM (Automatic Teller Machine) of a designated company (bank) and transfer from a registered bank account. The deferred payment setting is information indicating whether or not the setting for enabling electronic payment by deferred payment has been completed, and is set to either "completed" or "not completed." The deferred payment limit is the limit of deferred payment that can be used each month, the deferred payment usage amount is the amount of deferred payment that has already been used in the current month, and the deferred payment available amount is the amount of deferred payment that can be used in the current month, which is calculated by subtracting the deferred payment usage amount from the deferred payment limit. Although only one deferred payment limit is shown in the figure, in reality, there is also a daily upper limit, and the lower of these may be set as the deferred payment limit. Further details of deferred payment will be described later. The payment method setting is setting information indicating whether the user will perform electronic payment using the charge balance or payment by deferred payment at that time. The bank account and credit card number are information on a bank account or credit card number (account number, card number) that can be used to deposit funds into an electronic payment service. Charge history information is a history of the user's previous transfers to the electronic payment service to increase the charge balance. Payment history information is information that indicates the details of payments made by the user for each payment (date and time, store ID of the store where the purchase was made, payment amount, payment method, etc.).

[0028] 6 is a diagram showing an example of the contents of affiliated store / store information 176. The affiliated store / store information 176 includes, for example, a first table 176A in which an affiliated store ID and a store ID are associated with a store URL, a second table 176B in which an affiliated store name and sales amount (described above) are associated with an affiliated store ID, and a third table 176C in which a store ID is associated with a store name. In addition to this information, the affiliated store / store information 176 may also include information such as the category of the affiliated store or store, the location of the store, and payment patterns.

[0029] The information management unit 140 manages user information 172 and affiliated store / store information 176 based on information acquired from the user terminal device 10 and the second store terminal device 70. The information management unit 140 adds new records to, edits, and deletes the user information 172 and affiliated store / store information 176.

[0030] [Electronic payment] When payment information is acquired from the user terminal device 10 or the first store terminal device 50, the payment processing unit 130 refers to the user information 172 to acquire the "payment method setting" of the user. For a user whose "payment method setting" is set to "charge balance", the payment processing unit 130 performs electronic payment as follows. For example, the payment processing unit 130 performs electronic payment by decreasing the charge balance managed in association with the user ID and increasing the item value of the affiliated store's sales. The item value of the affiliated store's sales is not used as electronic money itself, for example, and an amount corresponding to the item value of the sales is transferred to a bank account in a cycle according to an agreement between the affiliated store and the electronic payment service.

[0031] The payment processing unit 130 performs electronic payment for users whose "setting information" is set to "deferred payment" as follows. Deferred payment is set separately from "credit payment" in cooperation with a credit card company, which is a separate entity from the operator of the electronic payment service, and the operator of the electronic payment service acts as a creditor and allows electronic payment that is not dependent on the charge balance within the deferred payment limit. In order to receive the deferred payment service, a credit card provided by the operator of the electronic payment service may be required. The amount used for deferred payment is settled on the payment date of the following month, for example, by debiting from a bank account, for one month. In this case, the payment processing unit 130 performs provisional payment by adding the payment amount to the deferred payment amount and subtracting the same amount from the available deferred payment amount, and when the closing date comes, it performs processing to debit the payment for the current month on the payment date of the following month as described above, or requests the operator of the credit card company to perform such processing. In addition, if the payment amount exceeds the available deferred payment amount at the time of provisional payment, an error notification is returned to the payment application 20.

[0032] [Payment app operation support function via voice (voice operation function)] Fig. 7 is a diagram showing an example of the functional configuration of a system that realizes the voice operation function of the payment application 20. Fig. 8 is a sequence chart showing an example of the process flow in which the payment application 20, the conversion server 200, and the payment server 100 work together to realize the voice operation function. Below, the flow of the sequence chart will be explained while explaining the contents of the process of each functional unit with reference to Fig. 7 as appropriate. It is assumed that at the start of the sequence chart, the user terminal device 10 is in a state in which the payment application 20 has already been started.

[0033] First, the user operates the payment application 20 to activate the voice input function, causing the payment application 20 to transition to a voice input waiting state (S21). The user speaks to the user terminal device 10 what they want to achieve using the payment application 20, thereby inputting the content of what they want to achieve as voice information to the payment application 20 (S22). It is assumed that the user terminal device 10 is equipped with a microphone for inputting this voice information. The payment application 20 has a voice input function 21, which is used to input the user's voice information and generate voice data (S23). The payment application 20 transmits the generated voice data to the conversion server 200 (S24).

[0034] Next, in the conversion server 200, the voice / text conversion unit 210 converts the voice data received from the user terminal device 10 into text data by performing processes such as acoustic analysis, phoneme extraction, and conversion of phonemes into words (S25). This conversion process (voice recognition process) may be realized using any voice recognition technology. Here, the voice / text conversion unit 210 is an example of an "acquisition unit."

[0035] Next, in the conversion server 200, the mapping information output unit 220 generates mapping information based on the text data generated by the voice / text conversion unit 210 (S26). The mapping information is information that enables the payment application 20 to recognize, by the command conversion function 22, a command to be executed in response to what the user wishes to achieve. In other words, the mapping information can be said to be information that maps what the user wishes to achieve to a command to be executed by the payment application 20. Here, the mapping information output unit 220 is an example of a "function identification unit", and a command is an example of a "function". Also, the voice input function 21 is an example of a "voice input unit", and the command conversion function 22 is an example of a "recognition unit".

[0036] Here, the mapping information may be any information that allows the payment application 20 to recognize the command to be executed. For example, the mapping information is represented by identification information of the command. In this case, the payment application 20 can recognize the command to be executed based on the identification information indicated by the mapping information. Also, for example, the mapping information may be represented by a command name (command string). In this case, the payment application 20 may execute the command string indicated by the mapping information as is. The mapping information may include parameters of the command as necessary.

[0037] For example, when it is recognized that a certain user wants to "pay for the purchase price of product B at store A with the charge balance," the mapping information may include identification information of store A (an example of an affiliated store), identification information of product B, and parameters such as the payment method along with a command string (which may be identification information) for the payment process. Furthermore, when the sales amount of product B is recognized in the conversion server 200, information on the payment amount may also be included as a parameter in the mapping information. In order to obtain information such as the sales amount, the conversion server 200 may be configured to communicate with, for example, the payment server 100 or the system of the affiliated store.

[0038] The mapping information output unit 220 may be configured as a conversational engine (e.g., a chatbot) that receives text data indicating what the user has said about what he or she wants to achieve, and returns command information for making the payment application 20 execute a command to achieve the desired result. Such a conversational engine may be constructed by an NLP / ML engine that combines a natural language understanding platform (NLP: Neuro Linguistic Processing) such as a dialogue flow, a machine learning algorithm (ML: Machine Learning) that learns the relationship between various functions (commands) possessed by the payment application 20 and matters realized by the various functions. The mapping information output unit 220 can specify a function to be executed by the payment application 20 to achieve the desired result of the user, for example, by inputting the text data into this conversational engine (an example of a learning result).

[0039] By being configured as such a conversational engine, the mapping information output unit 220 can flexibly and accurately recognize various requests of the user and generate command information for realizing the requests. By being provided with such command information, the payment application 20 can perform operations required to realize the requests expressed in various ways by the user. The mapping information output unit 220 transmits the generated mapping information to the user terminal device 10 (S27).

[0040] Next, in the user terminal device 10, the payment application 20 recognizes the command to be executed based on the mapping information received from the conversion server 200 by using the command conversion function (S28). The payment application 20 executes the command recognized from the mapping information, thereby executing a process to realize what the user wants to realize (S29). Here, the command executed by the payment application 20 may be one that can be completed by the payment application 20 alone (S29A), or may be one that works in cooperation with the payment server 100 (S29B).

[0041] According to the embodiment described above, it is possible to improve the voice operability of an application that provides an electronic payment function. In addition, such improved operability allows, for example, a person who is currently tied up or has poor eyesight to make payments by operating the payment application 20 by voice even with their hands free or their eyes closed.

[0042] <Modification> In the above embodiment, the case where the payment application 20 has the command conversion function 22 has been described, but the command conversion function 22 may be provided on the conversion server 200 side. In this case, the conversion server 200 recognizes a command to be executed by the payment application 20 by inputting the mapping information output by the mapping information output unit 220 to the command conversion function 22. The conversion server 200 may be configured to notify the recognized command to the payment application 20, and the payment application 20 may execute the command notified from the conversion server 200. Also, in this case, the mapping information output unit 220 may be configured integrally with the command conversion function 22, and configured to output a command to be executed by the payment application 20 in response to input of text data.

[0043] Payment application 20 may be configured to, when command conversion function 22 recognizes a command for realizing what the user wants to achieve based on mapping information supplied from conversion server 200, notify the user of the content of the command to be executed by voice before executing the command. To perform such notification, user terminal device 10 may be equipped with a speaker.

[0044] The conversion server 200 may be configured to obtain information required for executing a command from a user when executing the command to realize what the user wants to realize. For example, when the mapping information output unit 220 recognizes from the text data that there are multiple candidates for a command to be executed by the payment application 20, the mapping information output unit 220 may request the payment application 20 to provide supplemental information for determining which command to execute. Also, for example, when the mapping information output unit 220 recognizes from the text data a command to be executed by the payment application 20 but information to be a parameter of the command is insufficient, the mapping information output unit 220 may request the payment application 20 to provide supplemental information to compensate for the insufficient information. In this case, for example, the conversion server 200 generates a question for obtaining supplemental information and transmits it to the payment application 20, and the payment application 20 outputs the question by voice. The payment application 20 inputs the user's answer to the question by voice and transmits the voice data to the conversion server 200. The mapping information output unit 220 determines the command to be executed by the payment application 20 by selecting or supplementing a command based on the text data indicating the content of the user's answer. In this way, the conversion server 200 may be configured to recognize commands for realizing what the user has realized through dialogue with the user, thereby enabling the conversion server 200 to more accurately recognize commands for realizing what the user has realized.

[0045] The payment application 20 (or the conversion server 200) may have a voiceprint authentication function for authenticating a user by voice. The payment application 20 (or the conversion server 200) may be configured to provide a voice operation function to a user authenticated by voiceprint authentication.

[0046] The above describes the form for carrying out the present invention using an embodiment, but the present invention is not limited to such an embodiment, and various modifications and substitutions can be made within the scope that does not deviate from the gist of the present invention. [Explanation of symbols]

[0047] 10 User terminal device 20. Payment App 21 Voice input function 22 Command conversion function 50 First store terminal device 60 Store Code Image 70 Second store terminal device 72 Merchant Interface 100 Payment Server 110 Communications Department 120 Payment Contents Provider 130 Payment processing unit 140 Information Management Department 170 Storage section 172 User information 174 Payment Content Information 176 Affiliated Stores / Store Information 200 Conversion Server 210 Text conversion section 220 Mapping information output section

Claims

1. An information processing device that controls an operation of an application program running on a user's terminal device based on user's voice information input to the terminal device, comprising: an acquisition unit that acquires text data of voice information input to the terminal device, the voice information indicating the content of a speech regarding what the user wants to achieve by using the application program; a function identification unit that inputs the text data and identifies a function to be executed by the application program to realize what the user wants to realize, based on a learning result of learning the relationship between various functions of the application program and matters that can be realized by the various functions; An information processing device comprising:

2. the acquiring unit acquires voice data of the voice information from the terminal device, and acquires text data of the voice information by executing a voice recognition process on the voice data. The information processing device according to claim 1 .

3. the function identification unit inputs the text data into a conversational engine that combines a machine learning algorithm that learns the relationship with a natural language understanding platform, thereby identifying a function to be executed by the application program in order to realize what the user wants to achieve; The information processing device according to claim 1 .

4. when the text data lacks information for identifying a function to be executed by the application program, the function identification unit acquires supplemental information for filling in the lack of information from the terminal device by having an additional conversation with the user using the conversational engine; The information processing device according to claim 3 .

5. the application program cooperates with the payment management device to provide the user with an electronic payment service, and executes a function identified by the function identification unit to provide the user with an electronic payment service related to the identified function. The information processing device according to claim 1 .

6. 1. An information processing method in which a computer controls an operation of an application program running on a user's terminal device based on voice information of a user input to the terminal device, comprising: The computer, acquiring text data of the voice information input to the terminal device, the text data indicating the content of the speech related to what the user wants to achieve by using the application program; inputting the text data based on a learning result of learning the relationship between various functions of the application program and matters that can be realized by the various functions, and specifying a function to be executed by the application program in order to realize what the user wants to realize; Information processing methods.

7. A program for causing a computer to control the operation of an application program running on a user's terminal device based on voice information of the user input to the terminal device, comprising: The computer includes: acquiring text data of voice information input to the terminal device, the text data indicating the contents of a speech about what the user wants to achieve by using the application program; based on a learning result of learning the relationship between various functions of the application program and matters that can be realized by the various functions, inputting the text data and identifying a function to be executed by the application program in order to realize what the user wants to realize; Program for.

8. An information processing system comprising an application program that runs on a user's terminal device and a server device, the application program and the server device working together to control an operation of the application program based on voice information of a user input to the terminal device, The application program, a voice input unit for inputting voice information of the user and supplying voice data relating to the voice information to the server device; a recognition unit that recognizes a function to be executed by the application program in order to realize what the user wants to realize, based on the mapping information output from the server device; Equipped with The server device, an acquisition unit that acquires text data indicating the content of a speech regarding what the user wants to achieve by using the application program based on the voice data supplied from the application program; a function identification unit that inputs the text data and identifies a function to be executed by the application program to realize what the user wants to realize, based on a learning result of learning the relationship between various functions of the application program and matters that can be realized by the various functions; An information processing system comprising:

Images