Application program, service providing system, information processing method, and terminal device

Abstract

To provide a user with useful information.SOLUTION: An application program causes a computer of a terminal device to: acquire an image captured by an imaging unit; specify service information corresponding to feature information included in the image by referring to association information in which feature information included in the image stored in a storage unit is associated with service information related to an electronic settlement service; and superimpose the specified service information on the image captured by the imaging unit to be displayed on a display unit of the terminal device.SELECTED DRAWING: Figure 1

Description

[Technical field]

[0001] The present invention relates to an application program, a service providing system, and an information providing method. [Background technology]

[0002] 2. Description of the Related Art Conventionally, a support device has been disclosed that combines a real image with other information and displays the resulting image on a terminal device (see, for example, Patent Document 1). [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Patent No. 6704578 Summary of the Invention [Problem to be solved by the invention]

[0004] Conventional techniques may not be able to obtain information about features contained in an image, and may not be able to provide useful information to a user.

[0005] The present invention has been made in consideration of the above circumstances, and one of its objects is to provide an application program, a service providing system, and an information providing method that can provide useful information to users. [Means for solving the problem]

[0006] One aspect of the present invention is an application program that causes a computer of a terminal device to acquire an image captured by an imaging unit, identify service information corresponding to feature information contained in the image by referring to correspondence information stored in a memory unit that associates feature information contained in the image with service information related to electronic payment services, and superimpose the identified service information on the image captured by the imaging unit and display it on a display unit of the terminal device.

[0007] Another aspect of the present invention is an information provision method in which a computer of a terminal device acquires an image captured by an imaging unit, identifies service information corresponding to feature information contained in the image by referring to correspondence information stored in a memory unit that associates feature information contained in the image with service information related to electronic payment services, and superimposes the identified service information on the image captured by the imaging unit and displays it on a display unit of the terminal device. Effect of the Invention

[0008] According to one aspect of the present invention, it is possible to provide an application program, a service providing system, and an information providing method that can provide useful information to a user. [Brief description of the drawings]

[0009] [Figure 1] FIG. 1 is a diagram showing an example of a configuration for realizing an electronic payment service. [Diagram 2] This is a sequence diagram (part 1) illustrating the general flow of electronic payment. [Diagram 3] This is a sequence diagram (part 2) illustrating the general flow of electronic payment. [Figure 4] 1 is a configuration diagram of a payment server 100 according to a first embodiment. [Diagram 5] FIG. 13 is a diagram showing an example of the contents of user information 172. [Figure 6] FIG. 13 is a diagram showing an example of the contents of affiliated store / store information 176. [Figure 7] FIG. 13 is a diagram showing an example of the contents of first correspondence information 178A. [Figure 8] FIG. 13 is a diagram showing an example of the contents of second correspondence information 178B. [Figure 9] FIG. 1 shows an example of how the user terminal device 10 captures an image including the logo of an electronic payment service (the logo of a specific brand). [Figure 10] 2 is a diagram showing an example of content displayed on a display unit of the user terminal device 10. FIG. [Figure 11]13 is a diagram showing another example of the content displayed on the display unit of the user terminal device 10. FIG. [Figure 12] FIG. 13 is a diagram showing an example of how the user terminal device 10 captures an image including the logo of a store where electronic payment services are available. [Figure 13] 13 is a diagram showing another example of the content displayed on the display unit of the user terminal device 10. FIG. [Figure 14] 13 is a diagram showing another example of the content displayed on the display unit of the user terminal device 10. FIG. [Figure 15] 13 is a flowchart showing an example of a process flow related to an information provision service executed by the payment application 20 (information provision application 30) and the payment server 100. [Figure 16] 13 is a flowchart showing another example of the flow of processes related to the information provision service executed by the payment application 20 (information provision application 30) and the payment server 100. [Figure 17] 1 is a diagram showing an example of service information displayed on a display unit of a user terminal device 10. FIG. [Figure 18] 1 is a diagram showing an example of service information displayed on a display unit of a user terminal device 10. FIG. [Figure 19] FIG. 11 is a diagram showing an example of content for a user to make a payment using an electronic payment service. [Figure 20] 13 is a flowchart showing an example of a process for ordering a product, which is executed by the payment application 20 and the payment server 100. [Figure 21] FIG. 13 is a diagram showing another example of the correspondence information A. [Figure 22] FIG. 4 is a diagram showing an example of content displayed on a display unit. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0010] Hereinafter, with reference to the drawings, an embodiment of the application program, the service providing system, the server device, and the information providing method of the present invention will be described. The application program and the payment server cooperate to provide an electronic payment service. In the following description, the application program is referred to as a payment application. The electronic payment service is a service that supports payments related to the purchase of goods and services at a store. The store is, for example, a physical store (real store) existing in real space, but may also include a virtual store of electronic commerce. The virtual store may include a store provided by an entity different from the operator of the electronic payment service. In that case, when making a payment for shopping at the virtual store, the screen is controlled to transition to an interface screen of the electronic payment service. In the electronic payment service, the store is treated as belonging to, for example, an affiliated store (brand), and processing such as payment when a purchase is made at the store is mainly performed between the user and the affiliated store. Alternatively, processing such as payment may be performed between the user and the store.

[0011] [Electronic payment service] Fig. 1 is a diagram showing an example of a configuration for realizing an electronic payment service. The electronic payment service is realized mainly by a payment server 100. The payment server 100 communicates with, for example, one or more user terminal devices 10, one or more first store terminal devices 50, and one or more second store terminal devices 70 via a network NW. The network NW includes, for example, the Internet, a LAN (Local Area Network), a wireless base station, a provider device, and the like.

[0012] The user terminal device 10 is, for example, a portable terminal device such as a smartphone or a tablet terminal. The user terminal device 10 is a computer device having at least an optical reading function (for example, a camera), a communication function, a display function, an input reception function, a position identification function (for example, a position identification unit), and a program execution function. In the following description, configurations for realizing these functions are respectively referred to as a camera, a communication device, a touch panel, a self-position identification unit, a CPU (Central Processing Unit), and the like. In the user terminal device 10, a processor such as a CPU executes a payment application 20, thereby operating to provide an electronic payment service to a user in cooperation with a payment server 100. The payment application 20 is installed in the user terminal device 10 from, for example, an application store, and controls the camera, the communication device, the touch panel, and the like. The position identification unit identifies the position (position coordinates) of the user terminal device 10 using a Global Navigation Satellite System (GNSS).

[0013] The user terminal device 10 operates to provide an information provision service (described later) to the user in cooperation with the payment server 100 by executing the information provision application 30 by a processor such as a CPU. The information provision application 30 may be, for example, an application program included in the payment application 20, or may be a separate application program installed on the user terminal device 10 from an application store. The information provision application 30 controls a camera, a communication device, a touch panel, etc.

[0014] The first store terminal device 50 is installed, for example, in a store. The first store terminal device 50 is a computer device having at least a product price acquisition function, an optical reading function, a program execution function, and a communication function. The first store terminal device 50 includes a so-called POS (Point of Sale) device, and the product price acquisition function and the optical reading function may be realized by the POS device. The store code image 60 is placed in the store, and is a code image such as a QR code (registered trademark) printed on a paper or plastic medium. The store code image 60 may be displayed on a display placed in the store (which may be the display of a terminal device such as a smartphone).

[0015] The second store terminal device 70 is used by the operator of the affiliated store. The second store terminal device 70 is a smartphone, a tablet terminal, a personal computer, or the like. An interface 72 for affiliated stores runs on the second store terminal device 70. The interface 72 for affiliated stores may be an app for affiliated stores or a browser. The interface 72 for affiliated stores accepts coupon settings and the like made by the operator of the affiliated store and transmits them to the payment server 100. The second store terminal device 70, which is a smartphone, has the function of displaying a code image corresponding to a store code image and reading a code image displayed by the user terminal device 10 by executing the app for affiliated stores.

[0016] The payment server 100 realizes electronic payment based on payment information received from the user terminal device 10 or the first store terminal device 50. The first store terminal device 50 may include a POS device and an affiliated store server, in which case the payment information is sent from the POS device via the affiliated store server to the payment server 100. In the following explanation, no distinction is made between these two and it is assumed that the payment information is sent from the first store terminal device 50.

[0017] 2 and 3 are sequence diagrams illustrating the general flow of electronic payment. There may be two patterns of electronic payment: pattern 1 and pattern 2.

[0018] In the case of pattern 1 (hereinafter referred to as user scan) shown in FIG. 2, the user terminal device 10 with the payment application 20 activated decodes the store code image 60 by the optical reading function (S1). The store code image 60 includes store URL (Uniform Resource Locator) information. This store URL is an electronic payment service domain to which store-identifying information is added, and is associated with an affiliated store ID, a store ID, etc. in the payment server 100 (described later). The payment application 20 transmits the first payment information including the store URL and the account ID to the payment server 100 (S2). The payment server 100 searches for store information (described later) from the affiliated store ID and the store ID corresponding to the store URL, acquires the affiliated store name and the store name information (S3), and transmits it to the payment application 20 (S4). The user inputs the payment amount into the user terminal device 10 on the screen on which the affiliated store name and the store name are displayed (S5). Then, the user terminal device 10 generates second payment information including at least the payment amount, and transmits it to the payment server 100 (S6). The payment server 100 performs electronic payment based on the received second payment information (S7). The payment server 100 then transmits a payment completion notice (information for displaying a payment completion screen) to the payment application 20 (S8), and the payment application 20 displays the payment completion screen (S9). Note that when the store code image 60 is displayed on a display installed in the store, the store code image 60 may include not only the store URL but also information on the payment amount. In this case, the step of the user inputting the payment amount is omitted, and the information on the payment amount is included in the first payment information and transmitted to the payment server 100. Information on the affiliated store name and the store name may be included in the payment completion screen and displayed.

[0019] In the case of pattern 2 (hereinafter referred to as store scan) shown in FIG. 3, when the payment application 20 is started, when a payment operation is performed in the payment application 20, when an automatic update timing (e.g., every minute) occurs, and at other timings, the payment application 20 transmits a request for issuing a one-time code to the payment server 100 (S11). The payment server 100 generates a one-time code (S12) and transmits it to the payment application 20 (S13). The payment application 20 displays a code image such as a QR code or a barcode generated based on the one-time code (S14). The user holds (presents) the display surface of the user terminal device 10 over the first store terminal device 50, and the first store terminal device 50 decodes the code image by an optical reading function and obtains the one-time code, etc. (S15). The first store terminal device 50 then generates payment information including the one-time code, payment amount, affiliated store ID, store ID, etc., and transmits it to the payment server 100 (S16). The payment amount information is acquired in advance by reading a barcode or manually entering it. The payment server 100 identifies the user corresponding to the one-time code based on the received information and performs electronic payment (S17). The payment server 100 then transmits a payment completion notice to the payment application 20 (S18), and the payment application 20 displays a payment completion screen (S19).

[0020] Note that electronic payment may be performed using only one of the above patterns. Furthermore, the "account ID" described in FIG. 2 may be other information (e.g., a phone number) that can be used as user identification information. Furthermore, issuance of a one-time code may be omitted in the store scan, and the payment application 20 may display a code image generated based on the user's account ID. In this case, the payment server 100 identifies the user corresponding to the account ID instead of identifying the user corresponding to the one-time code.

[0021] [Payment server] FIG. 4 is a configuration diagram of the payment server 100 according to the first embodiment. The payment server 100 includes, for example, a communication unit 110, a payment content providing unit 120, a payment processing unit 130, an information management unit 140, a specification unit 142, an information providing unit 144, and a storage unit 170. The components other than the communication unit 110 and the storage unit 170 are realized by, for example, a hardware processor such as a CPU executing a program (software). Some or all of these components may be realized by hardware (including circuitry) such as an LSI (Large Scale Integration), an ASIC (Application Specific Integrated Circuit), an FPGA (Field-Programmable Gate Array), or a GPU (Graphics Processing Unit), or may be realized by cooperation between software and hardware. The program may be stored in advance in a storage device (a storage device having a non-transient storage medium) such as a hard disk drive (HDD) or a flash memory, or may be stored in a removable storage medium (a non-transient storage medium) such as a DVD or a CD-ROM, and may be installed in the storage device by mounting the storage medium in a drive device. Note that a part of the functional configuration included in the payment server 100 may be distributed to other devices.

[0022] The storage unit 170 is a HDD, a flash memory, a RAM (Random Access Memory), etc. The storage unit 170 may be a NAS (Network Attached Storage) device that the payment server 100 can access via a network. The storage unit 170 stores information such as user information 172, payment content information 174, affiliated store / shop information 176, and correspondence information 178. The correspondence information 178 includes first correspondence information 178A and second correspondence information 178B.

[0023] The communication unit 110 is a communication interface for connecting to the network NW. The communication unit 110 is, for example, a network interface card.

[0024] The payment content providing unit 120 has, for example, a function of a Web server, and provides information (content) for displaying various screens of the electronic payment service to the user terminal device 10. The payment content providing unit 120 appropriately reads necessary content from the payment content information 174 and provides it to the user terminal device 10. The user terminal device 10 accepts various inputs by the user while the content is being played by the payment application 20, and transmits the above-mentioned payment information and the like to the payment server 100.

[0025] The payment processing unit 130 performs payment processing based on the payment information transmitted by the user terminal device 10 or the first store terminal device 50. The payment processing unit 130 performs payment processing while referring to the user information 172.

[0026] FIG. 5 is a diagram showing an example of the contents of the user information 172. The user information 172 is an example of the registration information of a user. The user information 172 is, for example, a user URL, an account ID, a telephone number, a password, as well as information associated with an email address, a user ID, a name, an address, a date of birth, a registration date, a charge balance, a post-payment setting, a post-payment limit, a post-payment usage amount, a post-payment available amount, a payment method setting, a bank account, a credit card number, a charge history information, and a payment history information. The user URL is used for a remittance process between users. When registering for the electronic payment service, it is necessary to register a telephone number and a password. The account ID is issued to the user by the payment server 100, and the user ID is an ID that can be set by the user at will (does not have to be set). Similarly, the email address, and the name, address, and date of birth are information that can be set by the user at will (does not have to be set). The registration date is the date on which the user registered for the electronic payment service (the date on which the account was created). Hereinafter, the user's instance (electronic payment account) to which this information is associated will be referred to as an account.

[0027] The charge balance is information indicating the balance of electronic money that is set by a user by transferring money to an account in advance. The means of transfer include transfer from an ATM (Automatic Teller Machine) of a designated company (bank) and transfer from a registered bank account. The deferred payment setting is information indicating whether or not the setting for enabling electronic payment by deferred payment has been completed, and is set to either "completed" or "not completed." The deferred payment limit is the limit of deferred payment that can be used each month, the deferred payment usage amount is the amount of deferred payment that has already been used in the current month, and the deferred payment available amount is the amount of deferred payment that can be used in the current month, which is calculated by subtracting the deferred payment usage amount from the deferred payment limit. Although only one deferred payment limit is shown in the figure, in reality, there is also a daily upper limit, and the lower of these may be set as the deferred payment limit. Further details of deferred payment will be described later. The payment method setting is setting information indicating whether the user will perform electronic payment using the charge balance or payment by deferred payment at that time. The bank account and credit card number are information on a bank account or credit card number (account number, card number) that can be used to deposit funds into an electronic payment service. Charge history information is a history of the user's previous transfers to the electronic payment service to increase the charge balance. Payment history information is information that indicates the details of payments made by the user for each payment (date and time, store ID of the store where the purchase was made, payment amount, payment method, etc.).

[0028] 6 is a diagram showing an example of the contents of affiliated store / store information 176. The affiliated store / store information 176 includes, for example, a first table 176A in which an affiliated store ID and a store ID are associated with a store URL, a second table 176B in which an affiliated store name and sales amount (described above) are associated with an affiliated store ID, and a third table 176C in which a store ID is associated with a store ID. In addition to this information, the affiliated store / store information 176 may also include information such as the category of the affiliated store or store, the location of the store, and payment patterns.

[0029] The information management unit 140 manages user information 172 and affiliated store / store information 176 based on information acquired from the user terminal device 10 and the second store terminal device 70. The information management unit 140 adds new records to, edits, and deletes the user information 172 and affiliated store / store information 176.

[0030] The identification unit 142 identifies service information corresponding to the image or feature information included in the image acquired from the payment application 20. For example, the identification unit 142 identifies the service information by referring to the correspondence information 178 (details will be described later).

[0031] The information providing unit 144 provides the service information identified by the identifying unit 142 to the payment application 20.

[0032] [Electronic payment] When payment information is acquired from the user terminal device 10 or the first store terminal device 50, the payment processing unit 130 refers to the user information 172 to acquire the "payment method setting" of the user. For a user whose "payment method setting" is set to "charge balance", the payment processing unit 130 performs electronic payment as follows. For example, the payment processing unit 130 performs electronic payment by decreasing the charge balance managed in association with the user ID and increasing the item value of the affiliated store's sales. The item value of the affiliated store's sales is not used as electronic money itself, for example, and an amount corresponding to the item value of the sales is transferred to a bank account in a cycle according to an agreement between the affiliated store and the electronic payment service.

[0033] The payment processing unit 130 performs electronic payment for users whose "setting information" is set to "deferred payment" as follows. Deferred payment is set separately from "credit payment" in cooperation with a credit card company that is a separate entity from the operator of the electronic payment service, and the operator of the electronic payment service acts as a creditor and allows electronic payment that is not dependent on the charge balance within the deferred payment limit. In order to receive the deferred payment service, a credit card provided by the operator of the electronic payment service may be required. The amount used for deferred payment is settled on the payment date of the following month, for example, by debiting from a bank account, for one month. In this case, the payment processing unit 130 performs provisional payment by adding the payment amount to the deferred payment amount and subtracting the same amount from the deferred payment available amount, and when the closing date comes, it performs processing to debit the payment for the current month on the payment date of the following month as described above. In addition, if the payment amount exceeds the deferred payment available amount at the time of provisional payment, an error notification is returned to the payment application 20.

[0034] [Information provision service] The information provision application 30 (information provision application 30) and the payment server 100 cooperate to provide an information provision service. At this time, the payment server 100 provides the information provision application 30 with service information corresponding to the image acquired from the information provision service or the feature information included in the image, by referring to the first correspondence information 178A and / or the second correspondence information 178B. The information provision application 30 causes the display unit of the user terminal device 10 to display the provided service information. For example, the service information is displayed superimposed on a landscape image.

[0035] The landscape image is an image of a landscape captured by the camera of the user terminal device 10. The feature information included in the image is information indicating the features of a logo indicating the brand of the electronic payment service, or information indicating the features of the logos of stores such as affiliated stores and shops that can use the electronic payment service. The logo is, for example, a character, a figure, a pattern, or a combination of these. The logo is a so-called AR (Augmented Reality) marker.

[0036] Service information is information related to an electronic payment service. Information related to an electronic payment service includes information related to stores (affiliated stores or shops) where the electronic payment service can be used. For example, the service information is one or more of information on coupons that can be used at the electronic payment service (or at a specified affiliated store or shop), campaigns in the electronic payment service (lottery information, which will be described later, may also be included in the campaign information), or information on electronic stamp cards that can be used at the electronic payment service. A stamp card is a tool for providing a service in which stamps are given in accordance with electronic payments at a target store, and incentives are given to users when stamps are accumulated.

[0037] (First response information) The first correspondence information 178A is information for identifying service information corresponding to the logo of an electronic payment service, which is characteristic information included in an image. FIG. 7 is a diagram showing an example of the contents of the first correspondence information 178A. The first correspondence information 178A is information in which, for example, identification information, characteristic information of the logo of an electronic payment service, location information, and service information are associated with each other. In the first correspondence information 178A, information on shops located in the vicinity of the location where the logo of the characteristic information is located is set as service information. These shops are shops where the electronic payment service can be used.

[0038] (Second response information) The second correspondence information 178B is information for identifying service information corresponding to the logo of the store of the electronic payment service, which is characteristic information included in the image. FIG. 8 is a diagram showing an example of the contents of the second correspondence information 178B. The second correspondence information 178B is information in which, for example, identification information, characteristic information of the logo of the store of the electronic payment service, location information, and service information are associated with each other. In the second correspondence information 178B, information related to the store at the location where the logo of the characteristic information is present is regarded as service information. In the example of FIG. 8, information such as coupons, stamp cards, and campaigns is service information.

[0039] [Information provision service using first response information] Fig. 9 is a diagram showing an example of how the user terminal device 10 captures an image including a logo of an electronic payment service (a logo of a specific brand). As shown in Fig. 10, when the logo is captured, information (service information) of nearby stores where multiple (one or more) electronic payment services are available is displayed on the display unit of the user terminal device 10. For example, any information such as the name of the store, the store's logo, and the products handled by the store may be displayed. This service information is displayed superimposed on the captured landscape image.

[0040] When a user performs a predetermined operation such as a tap operation on an area corresponding to the store information, content including other information related to the store is displayed as shown in Fig. 11. For example, as shown in Fig. 11, content including the store's business hours and the store's menu may be displayed, or advantageous information for the user such as store coupons and campaigns may be displayed. In addition, information on the route from the location of the user terminal device 10 to the store in the map information, information indicating the location of the user terminal device 10 and the location of the store in the map information, etc. may be displayed on the display unit of the user terminal device 10.

[0041] [Information provision service using second response information] Fig. 12 is a diagram showing an example of how the user terminal device 10 captures an image including the logo of a store where an electronic payment service is available. As shown in Fig. 12, when the logo is captured, information about a plurality of stores (one or more) is displayed on the display unit of the user terminal device 10. For example, any information such as information about campaigns being held at the store, information about coupons that can be used at the store, and products handled at the store may be displayed.

[0042] When a user performs a predetermined operation, such as a tap operation, on an area corresponding to predetermined information, content corresponding to the information is displayed as shown in Fig. 13. The service information is displayed superimposed on the captured landscape image.

[0043] For example, when an area corresponding to a coupon is operated, content including detailed information about the coupon is displayed as shown in FIG. 14. Furthermore, the user can acquire a coupon in response to the operation. Acquiring a coupon means that coupon information is associated with the identification information of the user of payment application 20 so that the coupon can be used in payment application 20. This allows the user to use the coupon by performing a predetermined operation in electronic payment using payment application 20.

[0044] Note that the location information may be omitted from the first correspondence information 178A or the second correspondence information 178B. In this case, the service information corresponding to the feature information is provided without taking into account the location information.

[0045] [Flowchart (part 1)] 15 is a flowchart showing an example of a process flow related to an information provision service executed by the payment application 20 (information provision application 30) and the payment server 100. In this process, the payment application 20 (information provision application 30) acquires an image captured by a camera, transmits the acquired image or feature information included in the image to the payment server 100 (server device), and requests the payment server 100 to transmit service information corresponding to feature information included in an image specified by the payment server 100 with reference to correspondence information 178 in which the feature information is associated with service information related to an electronic payment service. In response to the request, the payment application 20 acquires the service information transmitted by the payment server 100, and displays the acquired service information on the display unit of the user terminal device 10 by superimposing it on the image captured by the camera.

[0046] First, the payment application 20 activates the camera based on the user's operation to capture an image (S100). Next, the payment application 20 transmits the captured image, the user's identification information, and the location information of the user terminal device 10 to the payment server 100 (S102).

[0047] Next, the payment server 100 acquires service information corresponding to the logo and location information based on the logo included in the image, the location information of the user terminal device 10, and the correspondence information 178 (S104). Next, the payment server 100 transmits the service information to the payment application 20 (S106). Next, the payment application 20 displays content in which the service information is superimposed on an image of a landscape on the display unit (S108).

[0048] When the user performs an operation on the service information, the payment application 20 accepts the operation on the service information (S110) and requests the payment server 100 to provide information corresponding to the operation (S112). Next, the payment server 100 acquires information corresponding to the request (S114) and provides the acquired information corresponding to the request to the payment application 20 (S116). Next, the payment application 20 displays the information provided by the payment server 100 on the display unit (S118). At this time, the provided information may be superimposed on a landscape image as shown in the above-mentioned Figs. 10 and 13. For example, the process executed by the payment application 20 is a function executed by the information providing application 30 included in the payment application 20.

[0049] As a result, when a user captures an image including a logo, service information corresponding to the logo is displayed on the display unit of the user terminal device 10. For example, a user can obtain useful information by capturing an image of a logo while walking down the street. In this way, it is possible to provide the user with information that is useful to them while improving entertainment value. Furthermore, the information provision service is also suitable for operators of stores and electronic payment services, as it serves as a promotion for the store or electronic payment service.

[0050] For example, in the process of S104, if the image does not include a logo, the payment server 100 may transmit information indicating that the logo is not included to the payment application 20, or may provide predetermined information such as information on stores that offer electronic payment services near the user's location or campaigns for electronic payment services. Also, if the image does not include a logo, the payment server 100 may provide the payment application 20 with service information corresponding to the location information of the user terminal device 10. In this case, for example, the correspondence information 178 includes information in which location information and service information are associated with each other.

[0051] Also, in S102, the payment application 20 transmits an image to the payment server 100, but instead (or in addition), an image in which the characteristics of a logo or an area including a logo is extracted from the image may be transmitted to the payment server 100. For example, the payment application 20 extracts the characteristics of a logo using a predetermined image processing algorithm or information indicating the characteristics of a logo stored in the storage unit of the user terminal device 10, extracts an area including a logo from the image, and transmits the characteristics of the logo or the extracted area including the logo to the payment server 100. In this way, if only a part of the captured image, rather than the entire image, or the results of processing the image (the characteristics of the logo) are transmitted to the payment server 100, the communication load is reduced.

[0052] Also, in S112, the payment application 20 sent a request for providing information to the payment server 100, but instead (or in addition) it may send a request to another server. The other server is, for example, a predetermined server device such as a server device corresponding to the store of the service information corresponding to the operation. For example, when the service information of "XX coffee" is operated, the payment server 100 requests the server device corresponding to "XX coffee" to provide the information. For example, the service information provided to the user is associated with, for example, information of the server device that is requested to provide the information in advance (for example, information of the access destination). The payment application 20 refers to this information and requests the other server device to provide the information.

[0053] [Flowchart (part 2)] 16 is a flowchart showing another example of the flow of processing related to the information provision service executed by the payment application 20 (information provision application 30) and the payment server 100. In this processing, the payment application 20 acquires the correspondence information 178 and identifies service information from a logo using the correspondence information 178. The payment application 20 (information provision application 30) acquires an image captured by a camera, identifies service information corresponding to the feature information included in the image by referring to correspondence information that associates feature information included in the image stored in the storage unit of the user terminal device 10 with service information related to electronic payment services, and displays the identified service information on the display unit of the user terminal device 10 by superimposing it on the image captured by the camera.

[0054] First, the payment application 20 acquires the corresponding information 178 from the payment server 100 or another server device (S200). Next, the payment application 20 activates the camera based on the user's operation to acquire an image (S202).

[0055] Next, the payment server 100 acquires service information corresponding to the logo and location information based on the logo included in the image, the location information of the user terminal device 10, and the correspondence information 178 (S204). Next, the payment server 100 displays content in which the service information is superimposed on the captured image of the scenery on the display unit (S206).

[0056] When the user performs an operation on the service information, the payment application 20 accepts the operation on the service information (S208) and requests the payment server 100 to provide information corresponding to the operation (S210). Next, the payment server 100 acquires information corresponding to the request (S212) and provides the acquired information corresponding to the request to the payment application 20 (S214). Next, the payment application 20 displays the information provided by the payment server 100 on the display unit (S216). At this time, the provided information may be superimposed on a landscape image as shown in the above-mentioned Figs. 10 and 13. For example, the process executed by the payment application 20 is a function executed by the information providing application 30 included in the payment application 20.

[0057] As a result, when a user captures an image including a logo, service information corresponding to the logo is displayed on the display unit of the user terminal device 10. In addition, in this process, since the payment application 20 identifies the service information, it is possible to provide the user with information that is useful to the user while reducing the communication load.

[0058] [Another example of service information (part 1)] The service information may be entertaining information such as a lottery. FIG. 17 is a diagram showing an example of service information displayed on the display unit of the user terminal device 10. For example, if the service information is a lottery, tapping an area corresponding to the lottery information causes the lottery to be animated and the winning result to be displayed. For example, if the service information is a logo of an electronic payment service, the lottery is a lottery for winning points for the electronic payment service or a special benefit such as a discount at a store where the electronic payment service can be used. For example, if the service information is a logo of a store that provides an electronic payment service, the lottery is a lottery for winning points for the store or a special benefit such as a discount at the store.

[0059] [Another example of service information (part 2)] The service information may be information for making a payment using an electronic payment service. The information for making a payment is, for example, information for guiding a user to a payment using an electronic payment service. For example, the information for guiding a user to a payment is information for ordering or reserving a product, service, or the like. For example, it is information for making a mobile order.

[0060] Fig. 18 is a diagram showing an example of service information displayed on the display unit of the user terminal device 10. For example, the display unit displays, as the service information, information for making a mobile order (or reservation) for an item at the logo store (an example of a "reception unit"). When the user operates the area for information for making a mobile order, the content shown in Fig. 19 is displayed.

[0061] FIG. 19 is a diagram showing an example of content for a user to make a payment using an electronic payment service. For example, this content includes information on the product to be purchased, the price of the product, information on the store where the product is purchased, the time until the product is provided, the payment method in the electronic payment service, and a button for confirming the payment. The time until the product is provided is obtained from the second store terminal device 70 of the store or a server device that manages store information. When the button for confirming the payment is operated, the payment is made and the product is ordered. The content shown in FIG. 19 is an example of "content related to the electronic payment, which is provided by requesting the server device to make the electronic payment and order for the product or service in response to the request."

[0062] [Product ordering flow chart] Fig. 20 is a flowchart showing an example of a process related to ordering a product, which is executed by the payment application 20 and the payment server 100. In this process, it is assumed that information for mobile ordering is displayed on the display unit as shown in Fig. 18 above.

[0063] When a user operates an area for information for mobile order, the payment application 20 transmits information indicating that the operation has been performed to the payment server 100 (S300). The payment server 100 identifies payment content (e.g., see FIG. 19) according to the operation (S302), and transmits the identified payment content to the payment application 20 (S304).

[0064] Next, the payment application 20 displays the transmitted payment content on the display unit (S306). When the user operates a button to confirm the payment in the payment content, the payment application 20 transmits payment instruction information to the payment server 100 instructing to confirm the payment (S308). Next, upon receiving the payment instruction information, the payment server 100 performs the payment (S310) and orders the product from the store according to the payment (S312). For example, the payment server 100 transmits product order information to the second store terminal device 70 of the store. As a result, the ordered product is prepared at the store.

[0065] Next, the payment server 100 provides the payment application 20 with content indicating that the payment and the order have been completed (S314). The payment application 20 displays the provided content on the display unit (S316).

[0066] As described above, the user can easily order products by manipulating the service information.

[0067] [Variation 1] The payment application 20 or the payment server 100 may determine the service information to be provided for each user. For example, the payment application 20 or the payment server 100 determines the service information based on the payment history of the user for the electronic payment service. For example, when providing service information corresponding to the logo of the electronic payment service, the member stores or stores where the user frequently makes electronic payments may be preferentially displayed on the display unit, or the member stores or stores where the user makes a large number of payments or the user makes a large amount of payments may be preferentially displayed on the display unit.

[0068] For example, correspondence information 178 may be prepared for each user, and service information may be provided by referring to correspondence information 178 corresponding to the user. Fig. 21 is a diagram showing another example of first correspondence information 178A. For example, as described above, service information is associated with characteristic information based on the payment history of the user.

[0069] [Variation 2] The payment application 20 or the payment server 100 may determine the service information to be provided for each user. For example, when providing service information corresponding to a store's logo, the information of the product or service for which the user has paid using the electronic payment service may be preferentially displayed on the display unit, or the product or service for which the user has paid many times or the product or service for which the user has paid a large amount may be preferentially displayed on the display unit. In the second modification, the payment server 100 is assumed to manage information of the product for which payment has been made as a payment history. For example, the payment server 100 is assumed to manage information of electronic payments (including information of purchased products or services) using the electronic payment service in mobile orders and online shopping.

[0070] For example, when an image including a store's logo is captured, the payment application 20 displays, on the display unit, information on a product that is frequently purchased at the store in the user's payment history as service information. Fig. 22 is a diagram showing an example of content displayed on the display unit. For example, when a user frequently places mobile orders for lunch set A and lunch set B at the store through electronic payment, the payment application 20 displays, on the display unit, service information for mobile orders for lunch set A and lunch set B.

[0071] As a result, when the user captures an image of the logo of an affiliated store or shop, information for mobile ordering is displayed on the display, and the user can easily order the product by operating the area for information for mobile ordering. Note that in the second modification, the process is described as being performed when an image of an affiliated store or shop logo is captured, but instead (or in addition), the process may also be performed when an image of an electronic payment service logo is captured. For example, information for ordering products that are frequently purchased by the user and that are located near the location of the user terminal device 10 is displayed on the display.

[0072] Instead of (in addition to) determining the service information to be displayed for each user as described above, the service information to be provided may be determined according to the environment the user is in. For example, on a hot day, a cold drink may be provided as the service information, and during the lunchtime, information on stores that serve lunch may be provided as the service information.

[0073] In the above embodiments, the information provision application 30 is included in the payment application 20, and the information provision application 30 executes processing together with the payment application 20. However, instead of (or in addition to) this, the information provision application 30 may be a separate application program not included in the payment application 20. In this case, the information provision application 30, the payment application 20, and the payment server 100 also cooperate to provide various services. Also, only the information provision application 30 may be provided to the user terminal device 10, and the information provision application 30 may run on the user terminal device 10.

[0074] In addition, the information provision application 30 has been described as being installed on a user terminal device 10 such as a smartphone, but instead (or in addition), the information provision service may be executed on a terminal device dedicated to executing the processing of the information provision application 30.

[0075] In the above example, the service information is identified from the logo included in the image, but instead of (in addition to) this, the service information may be identified from a distinctive object or display other than the logo. The distinctive object or display may be arbitrarily determined, for example, a distinctive building, an advertisement in the city, a person in the advertisement, or the like.

[0076] According to the embodiment described above, the payment application 20 can provide the user with useful information by providing the user with service information corresponding to the logo included in the image.

[0077] The above describes the form for carrying out the present invention using an embodiment, but the present invention is not limited to such an embodiment, and various modifications and substitutions can be made within the scope that does not deviate from the gist of the present invention. [Explanation of symbols]

[0078] 10 User terminal device 20. Payment App 30 Information App 100 Payment Server 120 Payment Contents Provider 142 Specific part 144 Information Provision Department 178 Supported Information

Claims

1. The terminal device computer acquiring an image including a logo of a store where a specific electronic payment service is available, the image being captured by the imaging unit; by referring to correspondence information that associates characteristic information of the logo of a store where the electronic payment service is available, which is included in the image and stored in a storage unit, location information, and store-related information provided by the store whose logo is available and which corresponds to the specific electronic payment service that corresponds to the location information, the characteristic information included in the image and the information related to the store that corresponds to the location information of the terminal device; superimposing information related to the identified store on the image captured by the imaging unit and displaying the information on the display unit of the terminal device; Application program.

2. The store-related information is information about coupons, campaigns, stamp cards, or products sold at the store. The application program according to claim 1 .

3. an application program according to claim 1 or 2; a server device that provides the correspondence information to the application program; A service providing system comprising:

4. The terminal device computer An image including a logo of a store where a specific electronic payment service is available is acquired, the image being captured by the imaging unit; identifying information related to the store that corresponds to the feature information included in the image and the location information of the terminal device by referring to correspondence information that associates feature information of the logo of the store where the electronic payment service is available, which is included in the image stored in a storage unit, location information, and information related to the store that is provided at the store whose logo is available and that corresponds to the specific electronic payment service that corresponds to the location information; superimposing information related to the identified store on the image captured by the imaging unit and displaying the information on the display unit of the terminal device; Information processing methods.

5. The information related to the store is information about coupons, campaigns, stamp cards, or products sold at the store. The information processing method according to claim 4.

6. An image including a logo of a store where a specific electronic payment service is available is acquired, the image being captured by the imaging unit; identifying information related to the store that corresponds to the feature information included in the image and the location information of the terminal device by referring to correspondence information that associates feature information of the logo of the store where the electronic payment service is available, which is included in the image stored in a storage unit, location information, and information related to the store that is provided at the store whose logo is available and that corresponds to the specific electronic payment service that corresponds to the location information; superimposing information related to the identified store on the image captured by the imaging unit and displaying the information on the display unit of the terminal device; Terminal device.

7. The store-related information is a coupon, a campaign, a stamp card, or information about products sold at the store. The terminal device according to claim 6.

Images