Application programs and payment systems

Abstract

To prevent a stagnation in electronic payments. [Solution] An application program that operates on a user's terminal device and cooperates with a payment server to provide electronic payment services to the user through communication with the payment server, the application program causing the terminal device to perform the following processes in response to the user's operation: displaying an image including a code image to be read by a store payment terminal when making a payment at a store; and outputting an output to guide the code image to move closer to a readable position on the store payment terminal.

Description

Technical Field

[0001] The present invention relates to an application program and a payment system.

Background Art

[0002] Conventionally, electronic payment services using an optical reading function have become widespread. In this electronic payment service, a store-side device may read a code image displayed by an application program operating on a terminal device. Usually, in this scenario, the user has to shade the display surface on which the code image is displayed towards the store-side device, making it difficult to visually recognize the content of the display surface. Therefore, it is difficult to confirm whether the code image is in an appropriate position with respect to the reading unit (camera or scanner) of the store-side device, and it is assumed that it may take time to read or the electronic payment may not be completed. In relation to this, an invention of an application program for receiving an electronic payment service has been disclosed (Patent Document 1). This application program displays an identification code on the display unit of the terminal device and enlarges or reduces the display size of the identification code in response to a specific operation when a specific operation by the user is received. [[ID=!]]

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] When the display side of the code image is held up to the store's device, if the display side is facing upwards and the store's device's reader reads the information from above, the user can see the display side. However, if the store's device's reader is oriented in a direction close to horizontal, the user may not be able to see the display side because they are holding the device up with the display side facing it. In this case, it is difficult for the user to confirm the position of the code image while holding the device up, and electronic payment may stall if the code image is not positioned within the readable range of the store's device.

[0005] This invention has been made in consideration of these circumstances, and one of its objectives is to provide an application program and a payment system that can suppress the stagnation of electronic payments. [Means for solving the problem]

[0006] One aspect of the present invention is an application program that operates on a user's terminal device and cooperates with a payment server to provide an electronic payment service to the user through communication with the payment server, the application program causing the terminal device to perform the following processes in response to the user's operation: displaying an image including a code image to be read by a store payment terminal when making a payment at a store; and outputting an output to guide the code image to move closer to a readable position on the store payment terminal. [Effects of the Invention]

[0007] According to one aspect of the present invention, it is possible to suppress the stagnation of electronic payments. [Brief explanation of the drawing]

[0008] [Figure 1] This diagram shows the basic forms of in-store electronic payment. [Figure 2] This diagram shows an example of a configuration for performing electronic payments (terminal payments) using a payment app. [Figure 3]This figure shows an example of the contents of user information 172. [Figure 4] This diagram shows an example of the contents of merchant / store information 174. [Figure 5] This diagram shows an overview of the processing flow when a user scan is performed. [Figure 6] This diagram shows an overview of the processing flow when a store scan is performed. [Figure 7] This diagram shows an example of a configuration for performing electronic payments (card payments) using payment cards. [Figure 8] This figure shows an example of the screen transitions in a payment application 20. [Figure 9] This figure shows an example of the enlarged display screen IM3. [Figure 10] This is a flowchart (part 1) showing an example of the processing flow performed by the payment application 20. [Figure 11] This is a flowchart (part 2) showing an example of the processing flow performed by the payment application 20. [Figure 12] This is a flowchart (part 3) showing an example of the processing flow performed by payment application 20. [Figure 13] This figure shows an example of a scenario where a code image is read. [Figure 14] This diagram shows an example of the processing flow performed by the payment application 20 and the payment server 100. [Figure 15] This figure shows an example of the relationship between the amount of positional displacement and the strength of the output. [Figure 16] This diagram illustrates the induction output. [Modes for carrying out the invention]

[0009] [overview] The following describes embodiments of the application program and payment system according to the present invention, with reference to the drawings. The application program, payment server, and credit card server work together to provide electronic payment services. In the following description, the application program will be referred to as the payment app. The electronic payment service is a service that supports payment for the purchase of goods and services at a store. A store is, for example, a physical store (real store) that exists in the real world, but may also include a virtual store for e-commerce. A virtual store may include one provided by an entity different from the operator of the electronic payment service. In that case, when settling a purchase at a virtual store, the user is controlled to transition to the interface screen of the electronic payment service. In the electronic payment service, a store is treated as belonging to, for example, a merchant (brand), and electronic payment when a purchase is made at a store is mainly made between the user and the merchant. Alternatively, electronic payment may be made between the user and the store.

[0010] [Types of in-store electronic payment methods] Figure 1 shows a basic configuration of in-store electronic payment. Basically, electronic payment is executed by three parties: a medium M held by the user U, store equipment E, and a payment system S. The medium M is a portable computer device such as a smartphone or a credit card. Store equipment E is present in a physical store (hereinafter simply referred to as "store") in the real world and includes POS devices, wireless communication devices, credit card readers, printed materials with code images such as barcodes or QR codes (registered trademarks), or display devices that show code images. A QR code is an example of a two-dimensional barcode. In in-store electronic payment, first, user identification information and payment amount information are shared unidirectionally or bidirectionally between the medium M and the store equipment E. During this process, one of the medium M or store equipment E optically reads various information from the code image displayed by the other, provides information via NFC (Near Field Communication), or reads the PAN (Primary Account Number) by a credit card reader. Then, either the medium M or the store equipment E (the one that obtains information from the other) transmits the payment information necessary for settlement to the payment system S via the network NW. Both the medium M and the store equipment E may also transmit some information to the payment system S. The payment system S manages various information of the user U and performs electronic payments between the store and the user U in various ways. Electronic payments are made using either a prepaid system or a post-paid system, or both, or by other methods. In addition, electronic payments may also include forms of online shopping, which are performed by both the user's terminal device and the payment system. The network NW includes, for example, the Internet, LAN (Local Area Network), wireless base stations, and provider equipment. The various devices that communicate via the network NW, as described below, are assumed to have communication devices such as network cards and wireless communication modules.

[0011] [Configuration (Terminal Payment)] FIG. 2 is a diagram showing an example of a configuration for performing electronic payment (terminal payment) using a payment application. This electronic payment is executed centering around a payment application 20 operating on a user terminal device 10 which is one of the media M, one or more store payment terminals 30 which are one of the store facilities E, one or more store code images 40, and a payment server 100 which constitutes a part of the payment system S. The payment server 100 communicates with the user terminal device 10, the store payment terminal 30, and one or more information terminals 50 via a network NW.

[0012] The user terminal device 10 is a portable terminal device such as a smartphone or a tablet terminal, for example. The user terminal device 10 is a computer device having at least an optical reading function, a communication function, a display function, an input reception function, and a program execution function. In the following description, the configurations for realizing these functions are respectively referred to as a camera, a communication device, a touch panel, a CPU (Central Processing Unit), etc. In the user terminal device 10, the payment application 20 is executed by a processor such as a CPU, and operates to provide an electronic payment service to the user in cooperation with the payment server 100. The payment application 20 is installed in the user terminal device 10 from, for example, an application distribution server (not shown), and controls the camera, the communication device, the touch panel, etc. of the user terminal device 10. In the following description, there may be a mixture of cases described as "transmitting information to the user terminal device 10 (or receiving / acquiring information from the user terminal device 10)" and cases described as "transmitting information to the payment application 20 (or receiving / acquiring information from the payment application 20)", but these are only differences in expression and are not intended to distinguish anything.

[0013] The store payment terminal 30 is installed in a store, for example. The store payment terminal 30 is a computer device (or an aggregate thereof) having at least a commodity price acquisition function, an optical reading function, a program execution function, and a communication function. The store payment terminal 30 includes a so-called POS (Point of Sale) device, and the POS device may have a commodity price acquisition function or an optical reading function.

[0014] The store code image 40 is placed in the store and is a code image such as a QR code printed on paper or plastic. The store code image 40 may also be displayed on a display placed in the store (this may also be the display of a terminal device such as a smartphone or tablet).

[0015] The information terminal 50 is used by the operator of a merchant that oversees the stores. In the electronic payment service, customers as providers of goods or services are treated as merchants (brands), and have one or more stores under their umbrella. There may also be merchants that operate only one store. The information terminal 50 is a smartphone, tablet, personal computer, etc. The information terminal 50 operates the merchant interface 55. The merchant interface 55 may be a merchant application or a web page displayed by a general-purpose browser. The merchant interface 55 accepts coupon settings etc. from the merchant operator and transmits them to the payment server 100. By executing the merchant interface 55, the information terminal 50 may have the function of displaying a code image corresponding to the store code image 40 or reading a code image displayed by the user terminal device 10 (in the latter case, an optical reading function is required).

[0016] The payment server 100 communicates with the credit card server 200 via a network NW. The payment server 100 includes, for example, a content provision unit 110, an information management unit 120, a payment processing unit 130, an estimation unit 140, a recognition unit 150, and a storage unit 170. The components other than the storage unit 170 are implemented, for example, by a hardware processor such as a CPU executing a program (software). Some or all of these components are LSIs (Large Scale Integrations), ASICs (Application Specific Integrated Circuits), FPGAs (Field-Programmable Graphite Arrays). The program may be implemented by hardware (including circuitry) such as a Gate Array or a GPU (Graphics Processing Unit), or by the collaboration of software and hardware. The program may be stored in advance on a storage device such as an HDD (Hard Disk Drive) or flash memory (a storage device equipped with a non-transient storage medium), or it may be stored on a removable storage medium such as a DVD or CD-ROM (a non-transient storage medium) and installed on the storage device when the storage medium is inserted into a drive device.

[0017] The storage unit 170 can be an HDD, flash memory, RAM (Random Access Memory), etc. The storage unit 170 may also be a NAS (Network Attached Storage) device that can be accessed by the payment server 100 via the network. The storage unit 170 stores information such as user information 172, merchant / store information 174, estimation model 176, and recognition model 178.

[0018] The content provider unit 110, for example, has the functionality of a web server and provides information (content) for displaying various screens of the electronic payment service to the user terminal device 10. The content provider unit 110 provides content to the user terminal device 10 in the form of a web page, or provides the user terminal device 10 with parameters necessary for the payment application 20 to render images.

[0019] The Information Management Department 120 edits, adds, and deletes user information 172 and merchant / store information 174, and manages these.

[0020] Figure 3 shows an example of the contents of User Information 172. User Information 172 is a collection of information such as User URL, Account ID, Phone Number, Password, Registration Date, Charge Balance, Electronic Money Type, Terminal Payment Method, Card Payment Method, Various History Information, Identity Verification Flag, Name, Address, Date of Birth, Email Address, Bank Account, Postpay Settings, and Postpay Conditions Information, all of which are linked to each other. Hereafter, the user instance (electronic payment account) to which this information is linked may be referred to as an account. In the figure, items indicated by "-" indicate that they are not set.

[0021] The user URL is used for transferring funds between users. When registering for the electronic payment service, registration of a phone number and password is required. The account ID is issued to the user by the payment server 100. The registration date is the date the user registered for the electronic payment service (the date the account was created). The charge balance is information indicating the balance of electronic money set by the user beforehand by sending money to the account. Methods of transfer include depositing money into an ATM (Automatic Teller Machine) of a designated company (bank) and transferring money from a registered bank account. The type of electronic money is information indicating, for example, whether the electronic money can be withdrawn or can only be used for electronic payments. The terminal payment method is setting information indicating whether the user will make an electronic payment using the charge balance (balance payment) or a deferred payment in terminal payments. The card payment method is setting information indicating whether the user will make an electronic payment using the charge balance (balance payment) or a deferred payment in card payments. Various types of history information include charge history, which is the history of when the user has previously sent money to the electronic payment service to increase the charge balance; payment history, which shows the details of each payment made by the user (date and time, store ID of the store where the purchase was made, merchant ID, payment amount, payment method, etc.); and usage history information of the payment app 20, which includes information on the history in which the enlarged code image (second image) described later was displayed. In addition to the information on the history in which the enlarged code image (second image) was displayed, the usage history information of the payment app 20 may include various other types of history information such as the usage history of various functions of the payment app 20, frequency of use, timing of use, and location of use (if the use of location information is permitted).

[0022] The "Verified" flag indicates whether the user has completed identity verification using an identification document. Post-payment becomes available only after identity verification is complete. In the diagram, the user with account ID "002" has not completed identity verification and therefore can only select balance payment as their terminal payment method. The bank account is the account number of a bank account into which funds can be deposited for the electronic payment service. The "Post-payment Settings" indicates whether the user has completed the necessary setup to enable post-payment. The "Post-payment Conditions" information shows various conditions for post-payment, such as the limit and the current month's usage amount.

[0023] Figure 4 shows an example of the contents of the merchant / store information 174. The merchant / store information 174 includes, for example, a first table 174A in which the merchant ID and store ID are associated with the store URL, a second table 174B in which the merchant name and sales amount (as described above) are associated with the merchant ID, and a third table 174C in which the store name is associated with the store ID. In addition to this information, the merchant / store information 174 may also include information such as the merchant or store category, the store's location, and payment patterns.

[0024] The payment processing unit 130 performs various processes for electronic payment. There are two methods for terminal payment, which are described below: the first method (user scan) and the second method (store scan).

[0025] Figure 5 shows an overview of the processing flow when a user scan is performed. First, the user terminal device 10, with the payment application 20 running, reads and decodes the store code image 40 using its optical reading function (S1). The store code image 40 contains information about the store URL. The payment application 20 sends first payment information, including the store URL and the user's account ID, to the payment server 100 (S2). The payment server 100 searches for merchant / store information 174 using the merchant ID and store ID corresponding to the store URL, obtains the merchant name and store name information (S3), and sends it to the payment application 20 (S4). The user enters the payment amount into the payment application 20 on the screen where the merchant name and store name are displayed (S5). Then, the payment application 20 generates second payment information, including at least the payment amount, and sends it to the payment server 100 (S6).

[0026] The payment processing unit 130 of the payment server 100 performs electronic payment based on the received second payment information if the "terminal payment method" in the user information 172 of the user is set to "balance payment" (S7-1). At this time, the payment processing unit 130 performs electronic payment by, for example, decreasing the charge balance managed in association with the user ID and increasing the item value of the merchant's sales proceeds. The item value of the merchant's sales proceeds is not used as electronic money itself, for example, but rather the amount corresponding to the item value of the sales proceeds is transferred to the bank account in a cycle according to the agreement between the merchant and the electronic payment service. On the other hand, if the "terminal payment method" is set to "post-payment", the payment processing unit 130 sends the first payment information and the second payment information to the credit card server 200 to request electronic payment (S7-2). The credit card server 200 performs electronic payment by adding the payment amount to the user's monthly usage amount based on the received information and deducting the monthly usage amount from the user's bank account after the closing date (S7-3).

[0027] Then, the payment processing unit 130 sends a payment completion notification (information for displaying the payment completion screen) to the payment application 20 via the content provision unit 110 (S8), and the payment application 20 displays the payment completion screen (S9). If the store code image 40 is displayed on a display placed in the store, the store code image 40 may include payment amount information as well as the store URL. In this case, the procedure for the user to enter the payment amount is omitted, and the payment amount information is included in the first payment information and sent to the payment server 100. Merchant name and store name information may be included and displayed on the payment completion screen.

[0028] Figure 6 shows an overview of the processing flow when a store scan is performed. First, when the payment app 20 is launched, when a payment operation is performed in the payment app 20, when it is time for an automatic update (for example, every minute), and at other times, the payment app 20 sends a request to the payment server 100 to issue a one-time code (S11). The payment processing unit 130 of the payment server 100 generates a one-time code (S12) and sends it to the payment app 20 (S13). The payment app 20 displays a code image such as a QR code or barcode that was generated based on the one-time code (S14). The user holds the display surface of the user terminal device 10 over the store payment terminal 30, and the store payment terminal 30 reads and decodes the code image using its optical reading function and obtains the one-time code, etc. (S15). Then, the store payment terminal 30 generates payment information including the one-time code, payment amount, merchant ID, store ID, etc., and sends it to the payment server 100 (S16). Payment amount information is obtained in advance through methods such as barcode scanning or manual entry.

[0029] The payment processing unit 130 of the payment server 100 identifies the user corresponding to the one-time code based on the received information, and if the "terminal payment method" in the user information 172 of that user is set to "balance payment", it performs electronic payment based on the received second payment information (S17-1). The content of the processing at this time is the same as the processing in S7-1 in Figure 5. On the other hand, if the "terminal payment method" is set to "post-payment", the payment server 100 sends the first payment information and the second payment information to the credit card server 200 to request electronic payment (S17-2). The credit card server 200 adds the payment amount to the user's monthly usage amount based on the received information and performs electronic payment by deducting the monthly usage amount from the user's bank account after the closing date (S17-3).

[0030] Then, the payment processing unit 130 sends a payment completion notification to the payment application 20 via the content provision unit 110 (S18), and the payment application 20 displays a payment completion screen (S19).

[0031] Furthermore, electronic payment may be performed using only one of the above patterns. Also, the "account ID" explained in Figure 2 may be other information that can be used as user identification information (for example, a phone number). In addition, the issuance of a one-time code may be omitted during store scanning, and the payment app 20 may display a code image generated based on the user's account ID. In that case, the payment server 100 will identify the user corresponding to the account ID instead of identifying the user corresponding to the one-time code.

[0032] Furthermore, instead of managing the "post-payment" settlement through the credit card server 200, it may be handled internally by the payment server 100. In this case, the configuration of the payment card 60, credit card server 200, etc., may be omitted.

[0033] [Payment Method (Card Payment)] Figure 7 shows an example of a configuration for electronic payment (card payment) using a payment card. This electronic payment is executed around a payment card 60, which is one of the media Ms; a credit processing terminal 70, which is one of the store equipment Es; and a payment server 100 and a credit card server 200, which constitute part of the payment system S. The credit card server 200 communicates with the credit processing terminal 70 via a network NW.

[0034] The credit processing terminal 70 is installed in the store, similar to the store payment terminal 30. The credit processing terminal 70 includes, for example, a credit payment terminal (credit card reader) and a POS device. The credit payment terminal reads the PIN (Personal Identification Number) from the inserted or scanned credit card and verifies it against the PIN entered by the user, or transmits the PAN (Primary Account Number) read from the credit card to the credit card server 200 via the POS device. The POS device works with the credit payment terminal to transmit information such as the payment amount to the credit card server 200. An acquisitioner server may be interposed between the credit processing terminal 70 and the credit card server 200, but for the sake of simplicity, the description of the acquisitioner server will be omitted below. The payment card 60 is, for example, similar in form to a commonly used credit card, with a communication chip embedded in the card base material. The communication chip contains a storage medium that stores the PIN and communicates with an external device via a contactor (or wireless antenna). Alternatively, the payment card 60 may be a magnetic stripe card. Note that the information (messages) transmitted and received when using a credit card includes an authorization message for authentication and a sales message to convey the payment amount; however, a detailed explanation distinguishing between these will be omitted below.

[0035] The credit card server 200 communicates with the settlement server 100 via a network NW. The credit card server 200 includes, for example, an information management unit 210, a credit interface 220, a settlement distribution unit 230, a credit settlement processing unit 240, and a storage unit 270. Components other than the storage unit 270 are implemented, for example, by a hardware processor such as a CPU executing a program (software). Some or all of these components may be implemented by hardware (including circuitry) such as an LSI, ASIC, FPGA, or GPU, or by the cooperation of software and hardware. The program may be stored in advance in a storage device such as an HDD or flash memory (a storage device with a non-transient storage medium), or it may be stored in a removable storage medium such as a DVD or CD-ROM (a non-transient storage medium) and installed in the storage device when the storage medium is mounted in a drive device. The storage unit 270 stores information such as card user information 272.

[0036] The information management unit 210 edits, adds, and deletes card user information 272 and manages it. Card user information 272 is information that associates, for example, information unique to the user (e.g., PAN), the card payment method, and the user's account ID (used by the payment server 100) with each other. The card payment method is setting information that indicates whether the user will make an electronic payment using their charged balance (balance payment) or a deferred payment in card payments.

[0037] The credit interface 220 determines whether the BIN (Bank Identification Number) in the PAN included in the message received from the credit processing terminal 70 is a code for the company. If it is a code for the company, it passes the message received from the credit processing terminal 70 to the settlement distribution unit 230. If it is not a code for the company, it discards the received message.

[0038] The settlement distribution unit 230 refers to the user's card user information 272 corresponding to the message obtained from the credit interface 220 and determines whether the "card payment method" is set to "post-payment". If the "card payment method" is set to "post-payment", the settlement distribution unit 230 notifies the credit interface 220 of this and passes the message obtained from the credit interface 220 to the credit payment processing unit 240. On the other hand, if the "card payment method" is set to "balance payment", the settlement distribution unit 230 adds the user's account ID to the message obtained from the credit interface 220 and sends it to the settlement server 100 to request electronic payment. The settlement server 100, upon receiving the request for electronic payment, performs the same processing as in S7-1 in Figure 5 and S17-1 in Figure 6.

[0039] The credit interface 220 checks the PAN and expiration date, and verifies whether the cumulative payment amount exceeds the monthly limit. The credit payment processing unit 240 adds the payment amount to the user's monthly usage amount based on the information contained in the message obtained from the payment distribution unit 230, and performs electronic payment by deducting the monthly usage amount from the user's bank account after the closing date.

[0040] [User support in store scanning (Part 1)] The following describes user support (Part 1) in store scanning. To address the difficulty in reading and verifying the information during store scanning as described above, the payment app 20 provides the following support functions. Figure 8 shows an example of the display screen transitions of the payment app 20. Upon startup, the payment app 20 displays the home screen IM1. In the home screen IM1, area A1 displays the first code image CD1, which is a barcode, and the second code image CD2, which is a two-dimensional barcode. Area A2, where icons etc. are displayed, and the "Pay" button B1 are displayed in a color corresponding to the currently selected fund source. Area A3 displays the available amount for each selectable fund source, but the available amount is masked depending on the user's selection. When the "Pay" button B1 is operated, the app transitions to the payment screen IM2. The payment screen IM2 is an example of the "first image" in the claims.

[0041] In the payment screen IM2, the first code image CD1 and the second code image CD2 are displayed enlarged in area A4 compared to the home screen IM1. Area A5, which is the background of area A4, is displayed in the same color as area A2 and the "Pay" button B1 of the home screen IM1. However, if the user has set multiple credit cards as fund sources, they cannot be distinguished by color, so information identifying the credit card is displayed in area A5. In area A6, the available amount corresponding to the currently selected fund source is selectively displayed. Button B2 is displayed, for example, after a predetermined time has elapsed since the start of displaying the payment screen IM2, as will be described later. Button B2 is an example of an image that proposes displaying an enlarged display screen IM3, which enlarges at least a part of the code image. When button B2 or any part of area A5 (an example of a predetermined part) is tapped, the enlarged display screen IM3, in which the code image is further enlarged, is displayed. The enlarged display screen IM3 is an example of the "second image" in the claims. When button B3 is operated, the camera of the user terminal device 10 is activated and switched to user scanning.

[0042] Figure 9 shows an example of the enlarged display screen IM3. In the enlarged display screen IM3, the first code image CD1 and the second code image CD2 are displayed in area A7 at a larger size compared to the payment screen IM2. Area A7 is displayed superimposed on the payment screen IM2. Here, since the scanning width of the barcode reader is generally uniform, the magnification ratio of the second code image CD2 compared to the payment screen IM2 is greater than the magnification ratio of the first code image CD1 compared to the payment screen IM2. It is preferable that the second code image CD2 is displayed at a size of 1 / 3 or more of the width of the display surface of the user terminal device 10. Note that when the "×" in the upper right corner of area A7 is operated, the user returns to the payment screen IM2.

[0043] In the payment screen IM2, the first code image CD1 and the second code image CD2 are enlarged compared to the home screen IM1. However, particularly with respect to the second code image CD2, if it is not directly aligned with the optical axis of the camera of the store payment terminal 30, or if the resolution of the camera of the store payment terminal 30 is low, reading may become impossible or take a long time. This can cause delays in electronic payments. In contrast, the above-described measures make it easier for the store payment terminal 30 to read the code, thereby suppressing delays in electronic payments.

[0044] Figure 10 is a flowchart (part 1) showing an example of the processing flow executed by the payment application 20. The processing in this flowchart starts when the display of the payment screen IM2 begins. First, the payment application 20 determines whether a predetermined time has elapsed since the display of the payment screen IM2 began (S100). If the predetermined time has not elapsed since the display of the payment screen IM2 began, the payment application 20 determines whether the payment has been completed based on whether or not a payment completion notification has been received from the payment server 100 (S102). If the payment has been completed, the processing in this flowchart ends; if the payment has not been completed, the process returns to S100. The predetermined time is, for example, a few seconds, and is a criterion for determining whether or not electronic payment is stalled based on the above situation.

[0045] If the payment application 20 determines in S100 that a predetermined time has elapsed since the start of displaying the payment screen IM2, it displays button B2, as described in Figure 8, on the user terminal device (S104). Next, the payment application 20 determines whether or not an area operation (tap operation at a predetermined location) has been performed (S106). If an area operation has been performed, the payment application 20 displays the enlarged display screen IM3 on the user terminal device 10 (S108). Regardless of whether the determination result in S106 is positive or negative, the payment application 20 determines whether or not the payment has been completed based on whether or not a payment completion notification has been received from the payment server 100 (S110). If the payment has been completed, the processing of this flowchart ends; if the payment has not been completed, the process returns to S106.

[0046] Alternatively, instead of following the above processing flow, if a predetermined amount of time has elapsed since the start of displaying the payment screen IM2, the display of button B2 may be skipped and the enlarged display screen IM3 may be automatically displayed.

[0047] Figure 11 is a flowchart (part 2) showing an example of the processing flow executed by the payment application 20. Note that the same steps as in Figure 10 are given the same step numbers, and further explanation is omitted. In this flowchart, if the payment application 20 determines in S100 that a predetermined time has elapsed since the start of displaying the payment screen IM2, it displays the enlarged display screen IM3 on the user terminal device 10 (S108). Then, the process proceeds to S102. This eliminates the need for the user to perform a series of actions, such as checking the display screen on the user terminal device 10 and then tapping button B2, thereby reducing the burden on the user. Note that the user may be able to choose whether to execute the flowchart in Figure 10 or Figure 11. Furthermore, the payment application 20 may switch between executing the flowcharts in Figure 10 and Figure 11 depending on environmental information such as the communication environment and the payment location. For example, if the communication environment is poor, or if the user is in a location with poor camera performance on the store payment terminal 30, the process in the flowchart of Figure 11 may be executed.

[0048] According to the process described above, appropriate support can be provided when a predetermined amount of time has elapsed since the start of displaying the payment screen IM2, that is, when electronic payment is actually stalled (or is presumed to be stalled).

[0049] The following describes the processing of the estimation unit 140 in the payment server 100. The estimation unit 140 collects usage history information of the payment application 20, including information on the history of when the enlarged display screen IM3 was displayed. The information on the history of when the enlarged display screen IM3 was displayed includes, for example, the percentage of times the enlarged display screen IM3 was displayed relative to the display of the payment screen IM2, as well as the number of data points, the time interval between dates, and other recognizable information. As mentioned above, the usage history information of the payment application 20 may include various history information such as the usage history of various functions of the payment application 20, usage frequency, usage timing, and usage location (if the use of location information is permitted). Based on the usage history information, the estimation unit 140 estimates whether there is a high probability that the enlarged display screen IM3 will be displayed in the situation where the payment screen IM2 was displayed, and sends information indicating the estimation result to the payment application 20.

[0050] The estimation unit 140 estimates whether the probability of the enlarged display screen IM3 being displayed is high for a given user by inputting information from the usage history of the payment app 20 into the estimation model 176 and obtaining the output. For example, the estimation model 176 is either a generative AI or a pre-trained model. If it is a generative AI, the estimation model 176 may be a function provided by an external generative AI server, in which case the estimation unit 140 performs the estimation by querying the generative AI server. If it is a pre-trained model, the estimation model 176 is trained using information from the usage history of multiple users as training data and whether each user performed the process to display the enlarged display screen IM3 as ground truth data, through processes such as backpropagation. Alternatively, the estimation unit 140 may perform the estimation process using a rule-based approach.

[0051] Alternatively (or in addition to) the above, the estimation unit 140 may refer to the output of the gyro sensor and acceleration sensor of the user terminal device 10 and estimate that there is a high probability that the enlarged display screen IM3 will be displayed when the vibration of the user terminal device 10 is large. The vibration of the user terminal device 10 is caused by the trembling of the user's hand or the location where the user terminal device 10 is located (for example, a shop on a ship). Similarly, the estimation unit 140 may estimate that there is a high probability that the enlarged display screen IM3 will be displayed when the user terminal device 10 is in a predetermined location such as on a ship.

[0052] If the payment application 20 obtains the estimation result from the estimation unit 140 and the estimation unit 140 estimates that there is a high probability that the enlarged display screen IM3 will be displayed, the payment application 20 will change the criteria for displaying the enlarged display screen IM3 to be more likely to be displayed compared to the case where there is no estimated high probability of display. For example, if the estimation unit 140 estimates that there is a high probability that the enlarged display screen IM3 will be displayed, the payment application 20 will shorten the predetermined time. In addition to this (or alternatively), if the estimation unit 140 estimates that there is a high probability that the enlarged display screen IM3 will be displayed, the payment application 20 may skip the display of button B2 and automatically display the enlarged display screen IM3.

[0053] Figure 12 is a flowchart (part 3) showing an example of the processing flow executed by the payment application 20. Note that the same steps as in Figure 10 are given the same step numbers and will not be explained again. In the processing of this flowchart, if it is determined in S100 that a predetermined time has elapsed since the start of displaying the payment screen IM2, the payment application 20 determines whether the target user is estimated by the estimation unit 140 of the payment server 100 to be a user with a high probability of being displayed on the enlarged display screen IM3 (S103). If it is estimated that the user is a user with a high probability of being displayed on the enlarged display screen IM3, the payment application 20 skips the display of button B2 and displays the enlarged display screen IM3 on the user terminal device 10 (S108).

[0054] The payment application 20 may display button B2 on the user terminal device 10 regardless of the elapsed time if the estimation unit 140 estimates that there is a high probability that the enlarged display screen IM3 will be displayed. Alternatively, the payment application 20 may display the enlarged display screen IM3 on the user terminal device 10 regardless of the elapsed time if the estimation unit 140 estimates that there is a high probability that the enlarged display screen IM3 will be displayed.

[0055] Furthermore, the payment application 20 may display the enlarged display screen IM3 on the user terminal device 10 when it detects an operation to change the position in which the user terminal device 10 is held, in addition to (or instead of) conditions such as a predetermined amount of time having elapsed since the start of display of the payment screen IM2 or the operation of button B2. An operation to change the position in which the user terminal device 10 is held can be detected, for example, by referring to the output of the gyro sensor or accelerometer of the user terminal device 10 and determining whether there is an output of a predetermined pattern. In this way, appropriate support can be provided by further limiting the scope of the situation.

[0056] Alternatively, the payment application 20 may have the functionality of the estimation unit 140. In this case, the payment application 20 will estimate, based on its own usage history, whether its users are likely to display the enlarged display screen IM3, similar to the estimation unit 140. In this case, the payment application 20 may have its own functionality equivalent to the estimation model 176, or it may perform the estimation by querying an external generation AI server or the like.

[0057] The process described above allows for appropriate support to be provided in situations where electronic payments are currently experiencing stagnation.

[0058] [User support in store scanning (Part 2)] The following describes user support (part 2) in store scanning. In response to the difficulty in confirming the reading during store scanning mentioned above, the payment application 20 provides the following support functions. Figure 13 shows an example of a scene in which a code image is read. In a store payment terminal 30 of this type, the reading unit 32 is almost upright and the optical axis direction is close to horizontal, so the user needs to hold the user terminal device 10 in a posture close to upright.

[0059] In the scenario shown in Figure 13, it is difficult for the user to see the code image displayed by the user terminal device 10. As a result, as mentioned above, there was a problem that electronic payments could stall because the code image was not positioned in a readable location on the store payment terminal 30 (i.e., a position directly facing the reading unit 32).

[0060] In contrast, the payment application 20 of the embodiment outputs information to guide the code image to move closer to the readable position of the store payment terminal 30, at least when the payment screen IM2 is displayed. Similar output may also be provided when the home screen IM1 is displayed in user support (part 2).

[0061] The readable position of the store payment terminal 30 is detected, for example, by activating the front camera (a camera facing the same direction as the display surface) of the user terminal device 10 to capture an image of the reading unit 32 of the store payment terminal 30, and based on the position in the captured image. At this time, the process of recognizing the reading unit 32 in the captured image is performed, for example, by the recognition unit 150 of the payment server 100. The captured image is transmitted to the payment server 100, for example, in the form of a video (live video stream).

[0062] The recognition unit 150 recognizes the position of the reading unit 32 in an image by inputting an image captured from the payment application 20 into the recognition model 178 and obtaining its output. The recognition unit 150 identifies an object that appears to be the scanner lens (reading unit 32) in real time from the live video stream of the front camera of the user terminal device 10. At this time, the recognition unit 150 analyzes the live video stream frame by frame and recognizes the position of the reading unit 32 in each frame by discovering specific objects or patterns. For example, the recognition model 178 is either a generative AI or a trained model. If it is a generative AI, the recognition model 178 may be a function provided by an external generative AI server, in which case the recognition unit 150 performs recognition by querying the generative AI server. If it is a pre-trained model, the recognition model 178 is trained using backpropagation and other processes, with training data consisting of images captured by the reading unit 32 in various forms under diverse lighting conditions, angles, and background noise (images captured in different store environments, including some or all of the lighting conditions, angles, and background noise), and the positions in those images as ground truth data. Note that the pre-trained model here may also include generative AI, and these can be collectively referred to as AI models. Alternatively, the recognition unit 150 may perform recognition processing by matching with multiple template images. Alternatively, the functionality of the recognition unit 150 may be provided by the payment application 20. In this case, the payment application 20 recognizes the position of the reading unit 32 in the captured image by performing processing equivalent to that of the recognition unit 150.

[0063] Figure 14 shows an example of the processing flow executed by the payment application 20 and the payment server 100. First, the payment application 20 determines whether the payment screen IM2 is displayed or not (S200). The determination process in S200 may be changed to "determine whether the home screen IM1 or the payment screen IM2 is displayed or not." If it is determined that the payment screen IM2 is displayed, the payment application 20 determines whether the user terminal device 10 is in an upright position or not (S202). An upright position is, for example, a position in which the angle with respect to the vertical direction of the display surface is approximately plus or minus 30 degrees. The payment application 20 obtains the position of the user terminal device 10 based on the output of the gyro sensor and accelerometer sensor equipped in the user terminal device 10 and makes the determination. Note that if the home screen IM1 is not included in the determination target in S200, the determination process in S202 may be omitted. If a negative determination result is obtained in S200 or S202, the process returns to S200.

[0064] If positive results are obtained in S200 and S202, the payment application 20 activates the front camera of the user terminal device 10 (S204) and sends the captured image to the payment server 100 to request recognition processing (S206). The recognition unit 150 of the payment server 100 performs the aforementioned recognition processing and sends (replies to) the recognition result (for example, whether the reading unit 32 was able to recognize the image, and if so, the center position of the reading unit 32 in the captured image) to the payment application 20 (S210).

[0065] The payment application 20 determines whether the reader 32 was recognized based on the recognition result (S212). If the reader 32 was not recognized, the process returns to S200. If the reader 32 was recognized, the payment application 20 obtains the amount of positional misalignment between the readable position of the store payment terminal 30 and the code image (S214). Details will be described later. The payment application 20 then outputs according to the amount of positional misalignment (S216). The payment application 20 determines whether the payment has been completed (S218). If the payment has not been completed, the process returns to S206; if it has been completed, the process of one cycle in this flowchart ends.

[0066] The positional misalignment amount will now be explained. The positional misalignment amount is, for example, the amount of misalignment in the image capture plane between the center position in the image of the reader 32 of the store payment terminal 30 and the display position of the code image (which the payment application 20 recognizes in advance). Alternatively, the positional misalignment amount may be any physical quantity as long as it has similar properties, such as the angle between the display position of the code image and the optical axis direction of the reader 32 of the store payment terminal 30. Furthermore, the positional misalignment amount may include the distance between the reader 32 of the store payment terminal 30 and the actual display position of the code image as an element. In the following explanation, the positional misalignment amount will be represented as the amount of misalignment in the image capture plane, that is, as a two-dimensional vector parallel to the display surface of the user terminal device 10 (a two-dimensional vector in a direction intersecting the optical axis direction of the reader of the store payment terminal).

[0067] Figure 15 shows an example of the relationship between the amount of misalignment and the strength of the output. For example, the payment application 20 uses the vibrator function of the user terminal device 10 to generate vibrations corresponding to the amount of misalignment. In the figure, Δx is the x component (horizontal component) of the misalignment, and Δy is the y component (vertical component) of the misalignment. As shown in the figure, the larger the absolute value of the misalignment, the stronger the vibrations (stronger the output) the payment application 20 generates. This relationship can also be reversed; the smaller the absolute value of the misalignment, the stronger the vibrations (stronger the output) the payment application 20 generates. In the figure, the region where the output is not zero is an example of a "predetermined positional relationship". Furthermore, the magnitude of the vibration can be adjusted by focusing on only one of the x or y components. Alternatively, instead of vibration, sound can be generated using a speaker with a similar tendency, or light can be generated using a light-emitting part that can be seen from the opposite side of the display surface of the user terminal device 10. This allows the payment application 20 to guide the code image so that it approaches a readable position in a direction (xy direction) that intersects with the optical axis direction of the reader unit 32 of the store payment terminal 30.

[0068] Figure 16 illustrates the induction output generated by the control described above. In the figure, the user terminal device 10 (shown only as an outline) moves from a position to the lower right relative to the center position 32A of the reading unit 32 toward the front. Accordingly, the vibration, which was initially strong, is changed to a medium / weak vibration. This allows the user to intuitively detect the correct position to hold the device over.

[0069] As described above, by informing users of the correct position to hold the device, delays in electronic payments can be prevented.

[0070] The payment application 20 and payment server 100 may implement only one of the user assistance features (1) and (2) for store scanning, or they may implement both. If both are implemented, they may be executed concurrently, or the user may be allowed to choose which one to execute. Alternatively, one of them may be automatically executed depending on the electronic payment situation.

[0071] Although embodiments for carrying out the present invention have been described above using examples, the present invention is not limited in any way to these embodiments, and various modifications and substitutions can be made without departing from the spirit of the present invention. [Explanation of Symbols]

[0072] E. Store facilities M medium S Payment System 10. User terminal device 20 Payment Apps 30 Store Payment Terminals 40 Store Code Images 60 Payment Cards 70 Credit card processing terminal 100 Payment Servers 130 Payment Processing Unit 140 Estimation part 150 Recognition part 176 Estimated Models 178 Recognition Models 200 credit card servers

Claims

1. An application program that operates on a user's terminal device and, through communication with a payment server, cooperates with the payment server to provide electronic payment services to the user, The aforementioned terminal device, The process involves displaying an image containing a code image that will be read by the store's payment terminal during payment at the store, in response to the user's actions. An output for guiding the code image to approach a readable position directly facing the reader of the store payment terminal, wherein the output intensity increases as the positional misalignment between the code image and the readable position increases, or decreases as the positional misalignment increases. Make it run, In the process of performing the output on the terminal device, A camera mounted on the same surface as the display surface of the terminal device is used to capture an image of the reading section of the store payment terminal, and the readable position of the store payment terminal is detected based on the position of the reading section in the captured image of the store payment terminal. Application program.

2. In the process of performing the output on the terminal device, With respect to the direction intersecting the optical axis of the reading unit of the store payment terminal, an output is provided to guide the code image so that it approaches the readable position. The application program according to claim 1.

3. In the process of performing the output on the terminal device, The captured image is input to a trained AI model to recognize the position of the reading unit, determine whether the code image and the readable position corresponding to the position of the reading unit are in a predetermined positional relationship, and if they are in the predetermined positional relationship, the AI ​​model outputs the result. The application program according to claim 1.

4. The aforementioned captured image is a moving image, In the process of performing the output on the terminal device, The trained AI model analyzes the aforementioned video image frame by frame to recognize the position of the reading unit. The application program according to claim 3.

5. The aforementioned trained AI model was trained using images captured by the reader of the store payment terminal, which were captured under different store environments, including some or all of the lighting conditions, angles, and background noise, as training data. The application program according to claim 3 or 4.

6. In the process of performing the output on the terminal device, The system generates vibration, sound, or light based on the relative relationship between the code image and the readable position of the store payment terminal. The application program according to claim 1.

7. The aforementioned terminal device, The orientation of the terminal device is acquired, When the terminal device is in an upright position, the process that produces the output is executed. The application program according to claim 1.

8. In the process of performing the output on the terminal device, The captured image is transmitted to the payment server, and the readable position of the store payment terminal is detected based on the position of the reading unit in the captured image returned from the payment server. The application program according to claim 1.

9. The application program described in claim 8, The payment server recognizes the position of the reading unit in the captured image and transmits the recognition result to the application program, Equipped with, Payment system.

10. The payment server recognizes the position of the reading unit by inputting the captured image into a trained AI model. The settlement system according to claim 9.

11. The aforementioned captured image is a moving image, The payment server recognizes the position of the reading unit by having the trained AI model analyze the video frame by frame. The payment system according to claim 10.

12. The aforementioned trained AI model was trained using images captured by the reader of the store payment terminal, which were captured under different store environments, including some or all of the lighting conditions, angles, and background noise, as training data. The payment system according to claim 10 or 11.

Images