Information processing device, information processing method and program

Abstract

To improve convenience of a user.SOLUTION: An information processing device comprises: a first acquisition unit that acquires an image of a physical medium to which a benefit is granted according to a set condition; a second acquisition unit that acquires media-related information regarding the media displayed on the physical medium acquired from the image; and a processing unit that applies the media-related information to media-related information of an electronic medium provided in an electronic payment service to generate the electronic medium or assist in the generation of the electronic medium.SELECTED DRAWING: Figure 4

Description

[Technical field]

[0001] The present invention relates to an information processing device, an information processing method, and a program. [Background technology]

[0002] 2. Description of the Related Art Conventionally, there has been disclosed an information processing device that notifies a user's information terminal of store guide information relating to a store when a condition is satisfied on an electronic stamp card (see, for example, Patent Document 1). [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Patent No. 7275347 Summary of the Invention [Problem to be solved by the invention]

[0004] In conventional technology, no consideration was given to adapting various physical media such as stamp cards and point cards to electronic media, resulting in low convenience for users.

[0005] The present invention has been made in consideration of the above circumstances, and an object of the present invention is to provide an information processing device, an information processing method, and a program that can improve convenience for users. [Means for solving the problem]

[0006] One aspect of the present invention is an information processing device comprising a first acquisition unit that acquires an image of a physical medium to which a benefit is granted according to set conditions, a second acquisition unit that acquires media-related information regarding a medium displayed on the physical medium obtained from the image, and a processing unit that applies the media-related information to media-related information of an electronic medium provided in an electronic payment service to generate the electronic medium or assist in the generation of the electronic medium. Effect of the Invention

[0007] According to one aspect of the present invention, it is possible to provide an information processing device, an information processing method, and a program that can improve user convenience. For example, it is possible to support a user in easily applying a physical medium to an electronic medium. [Brief description of the drawings]

[0008] [Figure 1] FIG. 1 is a diagram showing an example of a configuration for realizing an electronic payment service. [Diagram 2] This is a sequence diagram (part 1) illustrating the general flow of electronic payment. [Diagram 3] This is a sequence diagram (part 2) illustrating the general flow of electronic payment. [Figure 4] 1 is a configuration diagram of a payment server 100 according to a first embodiment. [Diagram 5] FIG. 13 is a diagram showing an example of the contents of user information 172. [Figure 6] FIG. 13 is a diagram showing an example of the contents of affiliated store / store information 176. [Figure 7] FIG. 13 is a diagram showing an example of an interface screen used to create a stamp card. [Figure 8] 2 is a diagram for explaining information input to a first model 320 and information output from the first model 320. FIG. [Figure 9] 13 is a diagram for explaining information input to a second model 420 and information output from the second model 420. FIG. [Figure 10] FIG. 13 is a diagram showing an interface screen IM2 for creating a stamp card to which the recognition result of the second recognition device 400 is applied. [Figure 11] 1 is a sequence diagram showing an example of a flow of processing executed by payment system 1. FIG. [Figure 12] 13 is a diagram for explaining information input to a third model 520 and information output from a third model 520. FIG. [Figure 13] FIG. 13 is a diagram for explaining stamp recommendations. [Figure 14] FIG. 11 is a diagram showing an example of the contents of correspondence information. [Figure 15] FIG. 2 is a sequence diagram showing an example of a process flow for generating an electronic stamp card using the user terminal device 10. [Figure 16] 13 is a diagram for explaining information input to a fourth model 620 and information output from the fourth model 620. FIG. [Figure 17] FIG. 13 is a diagram showing an example of an interface screen IM20 including an input field in which the number of stamps is set. [Figure 18] FIG. 13 is a sequence diagram showing an example of the flow of processing executed by the payment system 1 of the third embodiment. [Figure 19] 13 is a flowchart showing an example of the flow of processing executed by the payment server 100. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0009] [overview] The information processing device includes a first acquisition unit that acquires an image of a physical medium to which a benefit is granted according to set conditions, a second acquisition unit that acquires media-related information regarding the medium displayed on the physical medium obtained from the image, and a processing unit that applies the media-related information to media-related information of an electronic medium provided in an electronic payment service to generate the electronic medium or assist in the generation of the electronic medium.

[0010] Adaptation includes reflecting or proposing to reflect media-related information of physical media into electronic media-related information, setting or encouraging setting of media-related information of physical media into electronic media-related information, applying media-related information of physical media into electronic media-related information, etc.

[0011] The medium is, for example, a stamp card or a point card. The medium is not limited to the above, and may be other media provided by the store to the user. In the following embodiment, the medium is described as a stamp card. A stamp card is a card that is given a stamp each time the user meets the conditions set by the store (for example, each time a product or service is purchased), and when the number of stamps reaches a specified number, a special benefit is given to the user. In this embodiment, a service is provided that easily switches a physical stamp card to an electronic stamp card or supports the switching. Even in the case of other media such as a point card, a similar effect can be obtained by performing a process with a similar purpose.

[0012] The medium-related information is information indicated by characters or symbols written on the physical medium. The medium-related information includes information indicating the conditions for granting a privilege to a user (e.g., information such as 100 yen discount for 5 stamps), the expiration date of the medium (e.g., information such as valid for one year or valid until a certain date), information indicating the conditions for changing the achievement level regarding the granting of the privilege (e.g., the condition for stamping, such as granting one stamp for every 100 yen), or information regarding an area to which the information indicating the achievement level corresponds (e.g., the number of throttles). The information indicated by symbols is, for example, information other than characters contained in a stamp card, such as a stamp. The information indicated by symbols may be other information contained in a medium other than a stamp.

[0013] For example, a store manager may want to issue an electronic stamp card that can be used with an electronic payment service based on a physical stamp card. Also, a user who has a physical stamp card from a store may want to switch to an electronic stamp card issued by the store with an electronic payment service. In this embodiment, various processes are carried out to make this easy, as described below.

[0014] First Embodiment Hereinafter, an embodiment of a control method for an information processing device of the present invention will be described with reference to the drawings. An application program and a payment server cooperate to provide an electronic payment service. In the following description, the application program is referred to as a payment application. An electronic payment service is a service that supports payments related to the purchase of goods and services at a store. A store is, for example, a physical store (real store) existing in real space, but may also include a virtual store for electronic commerce. A virtual store may also include a store provided by an entity other than the operator of the electronic payment service. In that case, when making a payment for shopping at a virtual store, the screen is controlled to transition to an interface screen of the electronic payment service. In an electronic payment service, a store is treated as belonging to, for example, an affiliated store (brand), and processing such as payment when a purchase is made at a store is mainly performed between a user and the affiliated store. Alternatively, processing such as payment may be performed between a user and the store.

[0015] [Electronic payment service] Fig. 1 is a diagram showing an example of a configuration for realizing an electronic payment service. The electronic payment service is realized by a payment system 1 centered around a payment server 100. The payment server 100 communicates with, for example, one or more user terminal devices 10, one or more first store terminal devices 50, one or more second store terminal devices 70, a first recognition device 300, and a second recognition device 400, which are included in the payment system 1, via a network NW. The network NW includes, for example, the Internet, a LAN (Local Area Network), a wireless base station, a provider device, and the like.

[0016] The first recognition device 300 and the second recognition device 400 may be configured as a single device, or one or both of the functional configurations of the first recognition device 300 and the second recognition device 400 may be functional configurations included in the payment server 100.

[0017] The user terminal device 10 is, for example, a portable terminal device such as a smartphone or a tablet terminal. The user terminal device 10 is a computer device having at least an optical reading function, a communication function, a display function, an input reception function, and a program execution function. In the following description, components for realizing these functions are referred to as a camera, a communication device, a touch panel, a CPU (Central Processing Unit), etc. In the user terminal device 10, a processor such as a CPU executes a payment application 20, thereby operating to provide an electronic payment service to a user in cooperation with a payment server 100. The payment application 20 is installed in the user terminal device 10 from, for example, an application store, and controls the camera, communication device, touch panel, etc.

[0018] The first store terminal device 50 is installed, for example, in a store. The first store terminal device 50 is a computer device having at least a product price acquisition function, an optical reading function, a program execution function, and a communication function. The first store terminal device 50 includes a so-called POS (Point of Sale) device, and the product price acquisition function and the optical reading function may be realized by the POS device. The store code image 60 is placed in the store, and is a code image such as a QR code (registered trademark) printed on a paper or plastic medium. The store code image 60 may be displayed on a display placed in the store (which may be the display of a terminal device such as a smartphone).

[0019] The second store terminal device 70 is used by the operator of the affiliated store. The second store terminal device 70 is a smartphone, a tablet terminal, a personal computer, or the like. The second store terminal device 70 is, for example, a device equipped with an imaging unit. An interface 72 for affiliated stores runs on the second store terminal device 70. The interface 72 for affiliated stores may be an app for affiliated stores or a browser. The interface 72 for affiliated stores accepts coupon settings and the like by the operator of the affiliated store and transmits them to the payment server 100. The second store terminal device 70, which is a smartphone, has the function of displaying a code image corresponding to a store code image and reading a code image displayed by the user terminal device 10 by executing the app for affiliated stores.

[0020] The payment server 100 realizes electronic payment based on payment information received from the user terminal device 10 or the first store terminal device 50. The first store terminal device 50 may include a POS device and an affiliated store server, in which case the payment information is sent from the POS device via the affiliated store server to the payment server 100. In the following explanation, no distinction is made between these two and it is assumed that the payment information is sent from the first store terminal device 50.

[0021] 2 and 3 are sequence diagrams illustrating the general flow of electronic payment. There may be two patterns of electronic payment: pattern 1 and pattern 2.

[0022] In the case of pattern 1 (hereinafter referred to as user scan) shown in FIG. 2, the user terminal device 10 with the payment application 20 activated decodes the store code image 60 by the optical reading function (S1). The store code image 60 includes store URL (Uniform Resource Locator) information. This store URL is an electronic payment service domain to which store-identifying information is added, and is associated with an affiliated store ID, a store ID, etc. in the payment server 100 (described later). The payment application 20 transmits the first payment information including the store URL and the account ID to the payment server 100 (S2). The payment server 100 searches for store information (described later) from the affiliated store ID and the store ID corresponding to the store URL, acquires the affiliated store name and the store name information (S3), and transmits it to the payment application 20 (S4). The user inputs the payment amount into the user terminal device 10 on the screen on which the affiliated store name and the store name are displayed (S5). Then, the user terminal device 10 generates second payment information including at least the payment amount, and transmits it to the payment server 100 (S6). The payment server 100 performs electronic payment based on the received second payment information (S7). The payment server 100 then transmits a payment completion notice (information for displaying a payment completion screen) to the payment application 20 (S8), and the payment application 20 displays the payment completion screen (S9). Note that when the store code image 60 is displayed on a display installed in the store, the store code image 60 may include not only the store URL but also information on the payment amount. In this case, the step of the user inputting the payment amount is omitted, and the information on the payment amount is included in the first payment information and transmitted to the payment server 100. Information on the affiliated store name and the store name may be included in the payment completion screen and displayed.

[0023] In the case of pattern 2 (hereinafter referred to as store scan) shown in FIG. 3, when the payment application 20 is started, when a payment operation is performed in the payment application 20, when an automatic update timing (e.g., every minute) occurs, and at other timings, the payment application 20 transmits a request for issuing a one-time code to the payment server 100 (S11). The payment server 100 generates a one-time code (S12) and transmits it to the payment application 20 (S13). The payment application 20 displays a code image such as a QR code or a barcode generated based on the one-time code (S14). The user holds (presents) the display surface of the user terminal device 10 over the first store terminal device 50, and the first store terminal device 50 decodes the code image by an optical reading function and obtains the one-time code, etc. (S15). The first store terminal device 50 then generates payment information including the one-time code, payment amount, affiliated store ID, store ID, etc., and transmits it to the payment server 100 (S16). The payment amount information is acquired in advance by reading a barcode or manually entering it. The payment server 100 identifies the user corresponding to the one-time code based on the received information and performs electronic payment (S17). The payment server 100 then transmits a payment completion notice to the payment application 20 (S18), and the payment application 20 displays a payment completion screen (S19).

[0024] Note that electronic payment may be performed using only one of the above patterns. Furthermore, the "account ID" described in FIG. 2 may be other information (e.g., a phone number) that can be used as user identification information. Furthermore, issuing a one-time code may be omitted in the store scan, and the payment application 20 may display a code image generated based on the user's account ID. In this case, the payment server 100 identifies the user corresponding to the account ID instead of identifying the user corresponding to the one-time code.

[0025] [Payment server] FIG. 4 is a configuration diagram of the payment server 100 according to the first embodiment. The payment server 100 includes, for example, a communication unit 110, a content providing unit 120, a payment processing unit 130, an information management unit 140, a generating unit 150, and a storage unit 170. The components other than the communication unit 110 and the storage unit 170 are realized by, for example, a hardware processor such as a CPU executing a program (software). Some or all of these components may be realized by hardware (including circuitry) such as an LSI (Large Scale Integration), an ASIC (Application Specific Integrated Circuit), an FPGA (Field-Programmable Gate Array), or a GPU (Graphics Processing Unit), or may be realized by cooperation between software and hardware. The program may be stored in advance in a storage device (a storage device with a non-transient storage medium) such as an HDD (Hard Disk Drive) or flash memory, or may be stored in a removable storage medium (a non-transient storage medium) such as a DVD or CD-ROM, and installed in the storage device by inserting the storage medium into a drive device. These functional configurations may be distributed among multiple devices.

[0026] The storage unit 170 is a HDD, a flash memory, a RAM (Random Access Memory), etc. The storage unit 170 may be a NAS (Network Attached Storage) device that the payment server 100 can access via a network. The storage unit 170 stores information such as user information 172, payment content information 174, and affiliated store / shop information 176.

[0027] The communication unit 110 is a communication interface for connecting to the network NW. The communication unit 110 is, for example, a network interface card.

[0028] The content providing unit 120 has, for example, a function of a Web server, and provides information (content) for displaying various screens of the electronic payment service to the user terminal device 10. The content providing unit 120 appropriately reads out necessary content from the payment content information 174 and provides it to the user terminal device 10. The user terminal device 10 accepts various inputs by the user while the content is being played by the payment application 20, and transmits the above-mentioned payment information and the like to the payment server 100.

[0029] The payment processing unit 130 performs payment processing based on the payment information transmitted by the user terminal device 10 or the first store terminal device 50. The payment processing unit 130 performs payment processing while referring to the user information 172.

[0030] FIG. 5 is a diagram showing an example of the contents of the user information 172. The user information 172 is an example of the registration information of a user. The user information 172 is, for example, a user URL, an account ID, a telephone number, a password, as well as information associated with an email address, a user ID, a name, an address, a date of birth, a registration date, a charge balance, a post-payment setting, a post-payment limit, a post-payment usage amount, a post-payment available amount, a payment method setting, a bank account, a credit card number, charge history information, payment history information, and stamp card information. The user URL is used for a remittance process between users. When registering for the electronic payment service, it is necessary to register a telephone number and a password. The account ID is issued to the user by the payment server 100, and the user ID is an ID that can be set by the user at will (does not have to be set). Similarly, the email address, and the name, address, and date of birth are information that can be set by the user at will (does not have to be set). The registration date is the date on which the user registered for the electronic payment service (the date on which the account was created). Hereinafter, the user's instance (electronic payment account) to which this information is associated will be referred to as an account.

[0031] The charge balance is information indicating the balance of electronic money that is set by a user by transferring money to the account in advance. The means of transfer include transfer from an ATM (Automatic Teller Machine) of a designated company (bank) and transfer from a registered bank account. The deferred payment setting is information indicating whether or not the setting for enabling electronic payment by deferred payment has been completed, and is set to either "completed" or "not completed." The deferred payment limit is the limit of deferred payment that can be used each month, the deferred payment usage amount is the amount of deferred payment that has already been used in the current month, and the deferred payment available amount is the amount of deferred payment that can be used in the current month, which is calculated by subtracting the deferred payment usage amount from the deferred payment limit. Although only one deferred payment limit is shown in the figure, in reality, there is also a daily upper limit, and the lower of these may be set as the deferred payment limit. The payment method setting is setting information indicating whether the user will make electronic payment using the charge balance or by deferred payment at that time. The bank account and credit card number are information (account number, card number) of a bank account or credit card number that can be deposited into the electronic payment service, respectively. The charge history information is a history of the user's remittances to the electronic payment service to increase the charge balance. The payment history information is information showing the details of the payments made by the user (date and time, store ID of the store where the purchase was made, payment amount, payment method, etc.) for each payment.

[0032] The user stamp card information is information about a stamp card held by a user in an electronic payment service. The user stamp card information includes various information such as information for identifying the store of the stamp card, the number of electronic stamps on the stamp card, the expiration date of the stamp card, and the usage history of the stamp card.

[0033] FIG. 6 is a diagram showing an example of the contents of the affiliated store / store information 176. The affiliated store / store information 176 includes, for example, a first table 176A in which an affiliated store ID, a store ID, and store stamp card information are associated with a store URL, a second table 176B in which an affiliated store name and sales amount (described above) are associated with an affiliated store ID, and a third table 176C in which a store name is associated with a store ID. In addition to the above information, the affiliated store / store information 176 may include information on the category of the affiliated store or store, the location of the store, the payment pattern, and the like. The store stamp card information includes, for example, the number of throttles at which an electronic stamp is pressed, the conditions for giving a stamp, the conditions for giving a benefit, the contents of the benefit (such as a gift), and information on the expiration date. The store stamp card information is information set by the manager of the store.

[0034] The information management unit 140 manages user information 172 and affiliated store / store information 176 based on information acquired from the user terminal device 10 and the second store terminal device 70. The information management unit 140 adds new records to, edits, and deletes the user information 172 and affiliated store / store information 176.

[0035] The generating unit 150 includes, for example, a first acquiring unit 152, a second acquiring unit 154, and a processing unit 156. The following provides an overview of these functional units, and the details will be described later. The first acquiring unit 152 acquires an image (for example, an image of a physical stamp card) of a physical medium to which a privilege is granted according to a set condition. The second acquiring unit 154 acquires medium-related information (information displayed on the stamp card) regarding the medium displayed on the physical medium obtained from the image. The processing unit 156 applies the medium-related information to the medium-related information of the electronic medium provided in the electronic payment service to generate the electronic medium (for example, automatically generate it) or support the generation of the electronic medium (for example, support the input of information required for creation).

[0036] [Electronic payment] When payment information is acquired from the user terminal device 10 or the first store terminal device 50, the payment processing unit 130 refers to the user information 172 to acquire the "payment method setting" of the user. For a user whose "payment method setting" is set to "charge balance", the payment processing unit 130 performs electronic payment as follows. For example, the payment processing unit 130 performs electronic payment by decreasing the charge balance managed in association with the user ID and increasing the item value of the affiliated store's sales. The item value of the affiliated store's sales is not used as electronic money itself, for example, and an amount corresponding to the item value of the sales is transferred to a bank account in a cycle according to an agreement between the affiliated store and the electronic payment service.

[0037] The payment processing unit 130 performs electronic payment for users whose "setting information" is set to "deferred payment" as follows. Deferred payment is set separately from "credit payment" in cooperation with a credit card company, which is a separate entity from the operator of the electronic payment service, and the operator of the electronic payment service acts as a creditor and allows electronic payment that is not dependent on the charge balance within the deferred payment limit. In order to receive the deferred payment service, a credit card provided by the operator of the electronic payment service may be required. The amount used for deferred payment is settled on the payment date of the following month, for example, by debiting from a bank account, for one month. In this case, the payment processing unit 130 performs provisional payment by adding the payment amount to the deferred payment amount and subtracting the same amount from the available deferred payment amount, and when the closing date comes, it performs processing to debit the payment for the current month on the payment date of the following month as described above, or requests the operator of the credit card company to perform such processing. In addition, if the payment amount exceeds the available deferred payment amount at the time of provisional payment, an error notification is returned to the payment application 20.

[0038] [First recognition device] The first recognition device 300 includes, for example, a first recognition unit 310 and a first model 320. The first recognition unit 310 acquires character information (text) from an image using, for example, the first model 320, and provides the acquired information to the payment server 100. The first model 320 is, for example, a model that, when an image is input, outputs character information (for example, a collection of character information) included in the image.

[0039] [Second recognition device] The second recognition device 400 includes, for example, a second recognition unit 410 and a second model 420. The second recognition unit 410 acquires information (application information) to be applied to the generation of an electronic stamp card using, for example, the second model 420, and provides the acquired application information to the settlement server 100. The second model 420 is, for example, a model that outputs the application information included in character information when character information (for example, a collection of character information) is input. The first recognition device 300 and the second recognition device 400 will be described in detail later.

[0040] [Create stamp cards by administrator] A store manager (user) can create a stamp card to be used at the store by operating the affiliated store interface 72. FIG. 7 is a diagram showing an example of an interface screen used to create a stamp card. For example, the manager can create a stamp card by inputting information into interface screen IM1. Interface screen IM includes, for example, multiple input fields.

[0041] The input fields are fields for inputting, for example, the number of throttles at which an electronic stamp is to be applied, the conditions for granting one stamp, the conditions for granting a special benefit, the contents of the special benefit, the expiration date of the stamp card, etc. The administrator inputs information into these input fields and operates the decision button to confirm the contents of the electronic stamp card. For example, the processing unit 156 acquires the input information and generates a stamp card based on the acquired information. As a result, an electronic stamp card reflecting the input information is generated and provided to the user.

[0042] As described above, the administrator can create a stamp card. In this embodiment, in order to further improve the convenience of the administrator, the creation of the stamp card is supported. As described below, the generation unit 150 supports the creation of the stamp card by presetting information obtained from an image of a physical stamp card in an input field.

[0043] [Stamp card creation support (information acquisition)] For example, before displaying the interface screen IM1, the interface 72 for affiliated stores requests the administrator to provide an image of a physical stamp card in response to the administrator's operation, and once the image is provided by the administrator, the following processing is performed.

[0044] (Information acquired by the first acquisition unit) The first acquisition unit 152 of the payment server 100 acquires an image of a physical stamp card to which a special benefit is given according to a set condition. The image is, for example, an image captured by an administrator using an imaging unit provided in the second store terminal device 70, an image captured by another device, or the like, and is an image provided by the affiliated store interface 72.

[0045] (Information acquired by the second acquisition unit) The second acquisition unit 154 of the payment server 100 acquires information (medium-related information) displayed on the physical stamp card obtained from the image. The second acquisition unit 154 acquires, for example, medium-related information to be preset in an input field on a screen for creating a stamp card. The second acquisition unit 154 acquires medium-related information by using the first model 320 and the second model 420.

[0046] (Use of the first model) When an image provided by the first acquisition unit 152 is input, the first model 320 extracts a set of character information contained in the image. FIG. 8 is a diagram for explaining information input to the first model 320 and information output by the first model 320. When an image is input, the first model 320 outputs information in which the character information contained in the image is divided into sets of character information. For example, the first model 320 recognizes character information in the image using a known method such as OCR processing, and further identifies a set of character information that is estimated to have one meaning, and extracts the identified set of character information. For example, the first model 320 recognizes a set of character information by using natural language processing such as morphological analysis or by referring to the distance between adjacent or surrounding characters.

[0047] For example, in the example of FIG. 8, in the image IM-C in which a physical stamp card is captured, "one stamp for every 500 yen spent" is recognized as one set of character information. For example, this is because the distance between adjacent characters satisfies the criterion for being recognized as one set of character information. Note that the model for recognizing character information and the model for recognizing sets of character information may be separate models. Furthermore, the recognition of character information or sets of character information may be performed by a model in which learning data of combinations of correct answer data and images is learned by machine learning.

[0048] As described above, the set of character information is recognized by the first model 320. The first recognition unit 310 transmits the result of the recognition to the payment server 100.

[0049] (Use of the second model) When the second model 420 receives information of a set of character information provided by the second acquisition unit 154, it outputs information indicating input fields to be applied in creating a stamp card and information to be preset in the input fields. The input fields and the preset information are examples of application information. FIG. 9 is a diagram for explaining information to be input to the second model 420 and information output by the second model 420. The second model 420 is a model that outputs an intent and an entity when character information is input, for example. For example, the input field to be applied corresponds to the intent, and the preset information corresponds to the entity. The input fields (category of medium-related information) include, for example, input fields related to conditions for granting a privilege to a user, expiration date of the stamp card, conditions for stamping (conditions for changing the achievement level), or throttle number (area to which information indicating the achievement level is associated).

[0050] For example, when character information or a set of character information is input, the second model 420 performs natural language processing to identify one or more independent words from the character information. The second model 420 refers to information (learned information) prepared in advance to identify an input field corresponding to a word that is the same as or similar to the identified independent word. For example, when "There is no expiration date" is input, the second model 420 identifies the type of input field "Expiration date" corresponding to "Expiration date". For example, the second model 420 has been learned to set the type of input field to "Expiration date" when "Expiration date" or character information similar to "Expiration date" is input.

[0051] Furthermore, the second model 420 extracts an entity based on the results of prior learning. For example, when the second model 420 identifies the type of input field "expiration date", it identifies "not" or character information similar to "not", information indicating a deadline, and information similar to information indicating a deadline from the set of character information, and identifies the identified information as an entity. For example, the second model 420 identifies "not" included in the set of character information as an entity of "expiration date" which is the intent. As described above, the second model 420 has learned that "not" is an entity corresponding to an expiration date, and the expiration date indicated by the entity is "none", so it extracts the information of "none" preset in the above input field. The input field and the information preset in the input field are identified using the above method or a known method different from this.

[0052] The type of input field, which is the intent, is an example of a "category corresponding to medium-related information," and the preset information, which is the entity, is an example of "medium-related information." The second model 420 is an example of a model trained to output, when the character information is input, the medium-related information corresponding to the character information and the category (input field) of the medium-related information. Note that the second model 420 and the first model 320 may be integrated into one model.

[0053] As described above, the input fields and the preset information are recognized by the second model 420. The second recognition unit 410 transmits the recognition result to the payment server 100.

[0054] [Stamp card generation support (information preset)] In response to an instruction from an administrator of a member store of the electronic payment service, the second acquisition unit 154 acquires a combination of each of a plurality of pieces of medium-related information (information entered into an input field) included in the physical stamp card and a category (type of input field) corresponding to each of the plurality of pieces of medium-related information. The processing unit 156 presets each of the plurality of pieces of medium-related information in a setting field (input field) for setting the medium-related information of an electronic stamp card having a common category, generates an electronic stamp card in response to an operation by the administrator to provide a stamp card for approval or correction of the medium-related information preset in the setting field, and provides the generated electronic stamp card to the user terminal device 10 of the user in response to a request from the user who uses the electronic payment service.

[0055] The processing unit 156 acquires the result of the recognition by the second recognition device 400, and presets the acquired information in the corresponding input field. FIG. 10 is a diagram showing an interface screen IM2 for creating a stamp card to which the result of the recognition by the second recognition device 400 is applied. For example, the administrator provides an image to the settlement server 100, and the information of the physical stamp card is preset in the input field as shown in FIG. 10. The administrator can easily create a stamp card by confirming or correcting the preset information. For example, the administrator can easily create a stamp card by confirming the preset information and operating the decision button shown in FIG. 10.

[0056] Instead of (or in addition to) the preset, an electronic stamp card may be automatically generated based on information obtained from an image. For example, the payment server 100 may automatically set the contents of the input fields from an image to generate a stamp card. For example, in response to an instruction from an administrator of a member store of the electronic payment service, the second acquisition unit 154 acquires a combination of each of a plurality of medium-related information (information to be input into the input fields) included in a physical stamp card and a category (type of input field) corresponding to each of the plurality of medium-related information. The processing unit 156 may generate a stamp card in which each of the plurality of medium-related information is set to the medium-related information of an electronic stamp card having a common category, and provide the generated electronic stamp card to the user terminal device 10 of the user in response to a request from the user who uses the electronic payment service.

[0057] [Sequence diagram] 11 is a sequence diagram showing an example of the flow of processing executed by the payment system 1. This processing is executed, for example, when an administrator operates the member store interface 72 to start generation of an electronic stamp card.

[0058] First, the interface 72 for affiliated stores captures an image of the stamp card in response to an operation by the administrator (S100), and transmits the captured image to the payment server 100 (S102). The payment server 100 transmits the transmitted image to the first recognition device 300 (S104).

[0059] Next, the first recognition device 300 recognizes a set of character information from the image (S106) and transmits the recognition result to the payment server 100 (S108). Next, the payment server 100 requests the second recognition device 400 to recognize the type of the input field and the information to be preset together with the transmitted recognition result (S110). Next, the second recognition device 400 executes a recognition process using the transmitted recognition result of the first recognition device 300 (S112) and transmits the processing result to the payment server 100 (S114).

[0060] Next, the settlement server 100 presets information corresponding to the input fields of the stamp card creation screen using the processing result (type of input field and preset information) transmitted by the second recognition device 400 (S116), and provides the preset interface screen to the affiliated store interface 72 (S118). Next, the affiliated store interface 72 displays the interface screen with the information preset in the input fields on the display unit of the second store terminal device 70 (S120).

[0061] Through the above process, the payment server 100 can support the administrator in generating stamp cards. This allows the administrator to easily generate electronic stamp cards without inputting information on a physical stamp card.

[0062] The result of the recognition by the first recognition device 300 may be transmitted from the first recognition device 300 to the second recognition device 400. In addition, some of the processes in this process may be executed in another device. For example, the process of the first recognition device 300 or the second recognition device 400 may be executed in another device.

[0063] In the above embodiment, when a stamp card is generated, a product or service to which a stamp is to be applied may be specified, or a product or service to which a stamp is not to be applied may be specified. For example, the above specification is possible when a store having a POS device and linked to the payment server 100 generates a stamp card. In this case, the product or service to which a stamp is to be applied or the product or service to which a stamp is not to be applied is input in an input field from information included in the image (for example, a description indicating the target product written on a physical stamp card). When a user purchases a target product, the payment server 100 gives a stamp to an electronic stamp card. For example, identification information (for example, a JAN code) of a product related to the payment is transmitted from the POS device of the store to the payment server 100. The payment server 100 can determine whether the transmitted identification information of the product corresponds to a product to which a stamp is to be applied, and determine whether to apply the stamp.

[0064] According to the first embodiment described above, the payment server 100 acquires an image of a physical medium to which a benefit is granted according to set conditions, acquires media-related information about the medium displayed on the physical medium obtained from the image, and applies the media-related information to the media-related information of the electronic medium provided in the electronic payment service to generate electronic media or assist in the generation of electronic media, thereby improving the convenience of the administrator (user).

[0065] <Second embodiment> The second embodiment will be described below. In the first embodiment, character information is recognized from an image, and the creation of a stamp card is supported using the recognition result. In the second embodiment, symbols such as symbols and marks are used from an image to support the generation of a stamp card. A second acquisition unit 154 acquires a stamp from an image, and a processing unit 156 preferentially recommends a stamp having similar characteristics to the acquired stamp from among a plurality of stamps prepared in advance, or sets a stamp having similar characteristics as the stamp to be used on the stamp card. The following mainly explains the differences from the first embodiment.

[0066] The payment system 1 of the second embodiment includes a third model 520. The third model 520 may be provided in the payment server 100 or in another device. In this embodiment, the third model 520 is provided in, for example, the second recognition device 400.

[0067] When the image IM-C1 provided by the first acquisition unit 152 is input, the third model 520 outputs information indicating the characteristics of the stamp card included in the image. FIG. 12 is a diagram for explaining information input to the third model 520 and information output by the third model 520. The third model 520 is a model in which learning data, which is a combination of an image and information indicating the characteristics of the stamp included in the image, is learned. The third model 520 is a model trained to output the characteristics of the stamp included in the image when an image is input. For example, in the example of FIG. 12, when the image IM-C1 of a physical stamp card is input to the third model 520, the third model 520 outputs information indicating the characteristics of the flower stamp, which is the stamp pressed on the stamp card of the image IM-C1.

[0068] As described above, the characteristics of the stamp on the stamp card are recognized by the third model 520. The second recognition unit 410 transmits the result of the recognition to the settlement server 100.

[0069] For example, when creating a stamp card, the administrator can select a favorite stamp from among pre-prepared stamps as a stamp to be used in the electronic stamp card. At this time, the processing unit 156 preferentially recommends stamps having the same characteristics as or similar characteristics to those output by the third model 520 as a stamp to be used in the electronic stamp card.

[0070] FIG. 13 is a diagram for explaining stamp recommendations. An interface screen IM10 in FIG. 13 is a screen showing the form of stamps recommended when the third model 520 is not used. As shown in the interface screen IM10, stamps on a physical stamp card are not taken into consideration, so stamps that are not similar to stamps on a physical stamp card may be presented first. When an administrator wants to select a stamp similar to a stamp on a physical stamp card, the administrator needs to select the stamp by performing a predetermined operation such as scrolling the screen.

[0071] An interface screen IM11 in Fig. 13 is a screen showing the aspect of the stamp recommended when the third model 520 is used. As shown in the interface screen IM11, the characteristics of the stamp on the physical stamp card are taken into consideration, so that stamps similar to the stamp on the physical stamp card are presented first. For example, an electronic stamp with an image of a flower is preferentially recommended. The administrator can easily select a stamp similar to the stamp on the physical stamp card as a stamp to be used on the electronic stamp card.

[0072] For example, the processing unit 156 refers to the correspondence information to determine the stamp to be recommended with priority. Fig. 14 is a diagram showing an example of the contents of the correspondence information. The correspondence information is, for example, information in which the stamp features and the electronic stamp to be recommended are associated with each other. For example, the correspondence information specifies the electronic stamp to be recommended for the features output by the third model 520.

[0073] In the above example, a stamp is recommended, but instead of (or in addition to) this, the color or design of the stamp card may be recommended. For example, the second acquisition unit 154 acquires a feature, which is the color or design of the stamp card, from an image, and the processing unit 156 may preferentially recommend an electronic stamp card having a feature similar to the acquired feature from among a plurality of designs prepared in advance, or may set an electronic stamp card having a similar feature to be used. In this case, when an image is input, a learning model that outputs information indicating the color or design feature of the stamp card included in the image is used. This model is, for example, a model in which learning data that is a combination of an image and information indicating the feature of the stamp card (information indicating the color or design feature of the stamp card) is learned. This model is a model that is trained to output the feature of the stamp card included in the image to which the image is input.

[0074] According to the second embodiment described above, the payment server 100 can improve user convenience by acquiring the characteristics of the stamp card from the image and recommending to the user a stamp or stamp card having characteristics similar to the acquired characteristics of the stamp card, or generating a stamp or stamp card having similar characteristics.

[0075] <Third embodiment> The third embodiment will be described below. In the first and second embodiments, the administrator generates the electronic stamp. In the third embodiment, an example will be described in which a user who uses an electronic payment service to make a purchase generates an electronic stamp card. The differences from the first and second embodiments will be described below.

[0076] In the third embodiment, a user generates an electronic stamp card using a user terminal device (payment application 20 or another application). The other application may be, for example, a mini application included in the payment application.

[0077] In response to a request from a user who uses the electronic payment service to make an electronic payment at a store to purchase a product or service, the second acquisition unit 154 acquires a combination of each of the plurality of medium-related information included in the physical stamp card and a category corresponding to each of the plurality of medium-related information. The processing unit 156 generates an electronic medium by setting each of the plurality of medium-related information acquired by the second acquisition unit 154 to the medium-related information of an electronic medium having a common category. The processing unit 156 associates the generated electronic medium with the user's identification information managed by the electronic payment service, and sets the electronic medium to a state in which it can be used in the electronic payment service. For example, the processing unit 156 associates the generated stamp card with the user stamp card information of the user information 172, and makes it available.

[0078] [Sequence diagram] Fig. 15 is a sequence diagram showing an example of the process flow for generating an electronic stamp card using the user terminal device 10. In Fig. 15, the member store interface 72 in Fig. 11 described above has been replaced with the user terminal device 10. Other processes are similar, so the same step numbers are given. The process executed by the member store interface 72 in Fig. 11 is executed by the user terminal device 10 in Fig. 15. As a result, information on the physical stamp card is preset in the input field of the interface screen that is displayed when generating an electronic stamp card.

[0079] At this time, the number of stamps on the physical stamp card may be reflected as follows: For example, when the fourth model 620 receives an image provided by the first acquisition unit 152, it outputs information indicating the number of stamps included in the image.

[0080] The payment system 1 of the third embodiment includes a fourth model 620. The fourth model 620 may be provided in the payment server 100 or in another device. In this embodiment, the fourth model 620 is provided in the second recognition device 400, for example.

[0081] FIG. 16 is a diagram for explaining information input to the fourth model 620 and information output by the fourth model 620. The fourth model 620 is a model in which learning data, which is a combination of an image and the number of stamps included in the image, is learned. The fourth model 620 is a model trained to output information indicating the number of stamps included in the image when an image is input. For example, in the example of FIG. 16, when an image IM-C2 of a physical stamp card is input to the fourth model 620, the fourth model 620 outputs information indicating the number of stamps pressed on the stamp card of the image.

[0082] The processing unit 156 acquires information indicating the number of stamps output by the fourth model 620, and sets the number of stamps in an input field for inputting the number of stamps on the interface screen for generating a stamp card. FIG. 17 is a diagram showing an example of an interface screen IM20 including an input field in which the number of stamps is set. As shown in FIG. 17, the interface screen IM20 is provided with an input field for inputting the number of stamps pressed on a physical stamp card. The processing unit 156 sets the number of stamps obtained from the image in the input field. This number of stamps is a setting that cannot be changed or input by, for example, the user.

[0083] According to the third embodiment described above, the settlement server 100 can easily generate an electronic stamp card from an image of a physical stamp card. Furthermore, since the processing unit 156 applies the number of stamps obtained from the image to the number of stamps on the electronic stamp card, an electronic stamp card that reflects the information on the physical stamp card can be easily generated.

[0084] <Fourth embodiment> The fourth embodiment will be described below. In the third embodiment, the user did not need the administrator's consent when generating an electronic stamp card. In the fourth embodiment, the electronic stamp card is generated on the condition of the administrator's consent. The processing unit 156 may acquire information permitting the electronic stamp card to be set to a usable state from a terminal device (e.g., the affiliated store interface 72) of the manager of the store that provides the electronic medium, or may set the electronic stamp card to a usable state when the information is acquired. The differences from the third embodiment will be described below.

[0085] [Sequence] Fig. 18 is a sequence diagram showing an example of the flow of processing executed by the payment system 1 of the third embodiment. For example, the processing of Fig. 15 described above is executed separately from the processing of this sequence diagram.

[0086] First, the user terminal device 10 transmits an instruction to generate an electronic stamp card to the payment server 100 in response to the user's operation (S200). Next, the payment server 100 requests the affiliated store to approve the generation of the electronic stamp card (S202). Next, when the administrator performs an operation of approval (S204), the affiliated store interface 72 transmits approval information to the payment server 100 (S206). Next, the payment server 100 completes the generation of the electronic stamp card in response to obtaining the approval information (S208). Note that the timing of the above approval request may be different from the above.

[0087] For example, when a user visits a store and presents a physical stamp card, the administrator informs the user that an electronic stamp card can be issued (that it is possible to switch). At this time, the user uses the user terminal device 10 to switch from the physical stamp card to an electronic stamp card. For example, by the process of S202 described above, an image of the captured physical stamp card and information for identifying the user are displayed on the display unit of the second store terminal device 70. By referring to this information, consenting to the generation described above and collecting the physical stamp card, the coexistence of physical stamp cards and electronic stamp cards is prevented.

[0088] According to the fourth embodiment described above, the settlement server 100 allows the user to change the physical stamp card to an electronic stamp card on the condition of the administrator's consent. This prevents the physical stamp card and the electronic stamp card from coexisting.

[0089] <Fifth embodiment> The fifth embodiment will be described below. In the third and fourth embodiments, it has been described that electronic stamp cards are provided at stores that provide physical stamp cards. In the fifth embodiment, processing is performed that takes into account cases where electronic stamp cards are not provided. The differences from the third embodiment will be described below.

[0090] When a user performs an operation to generate an electronic stamp card based on a physical stamp card, the payment server 100 executes processing according to the determination result as to whether or not electronic stamp cards are provided at the store.

[0091] [Sequence] FIG. 19 is a flowchart showing an example of the flow of the process executed by the payment server 100. First, the payment server 100 judges whether or not the user of the user terminal device 10 has performed an operation to create an electronic stamp card (S300). If the operation has been performed, the payment server 100 specifies the store that will generate the electronic stamp card (S304). The store may be specified, for example, based on (1) the user's operation, or based on (2) information obtained from an image of a physical stamp card. In the case of (1), for example, when generating an electronic stamp card, the user specifies the store corresponding to the stamp card. As a result, information for specifying the store, such as the store's identification information, is provided from the user terminal device 10 to the payment server 100. In the case of (2), for example, the payment server 100 specifies the store name, the store's phone number, and the store's URL included in the image, and specifies the store by referring to the specified information and information related to the store that is held in advance. In the case of (2), for example, an image of a physical stamp card is provided from the user terminal device 10.

[0092] Next, the payment server 100 judges whether the specified store issues an electronic stamp card (S304). If the specified store issues an electronic stamp card, the payment server 100 provides the content (interface screen) for generating the electronic stamp card to the user terminal device 10 (S306).

[0093] If the identified store does not issue an electronic stamp card, the payment server 100 provides the user terminal device 10 with information that an electronic stamp card has not been issued (S308). Next, the payment server 100 notifies the store of information indicating that a request for the creation of a stamp card has been received (S310). For example, the notification is sent via the affiliated store interface 72. For example, the payment server 100 may send the above notification to the store when there are more requests than a threshold.

[0094] According to the fifth embodiment described above, the payment server 100 notifies a store that has not issued a stamp card that a user has requested issuance of the stamp card, which can improve the motivation of the store to issue an electronic stamp card. This can be expected to accelerate issuance of stamp cards and to further popularize stamp cards, improving the convenience of users.

[0095] According to the embodiment described above, the payment server 100 acquires an image of a physical medium to which a benefit is granted according to set conditions, acquires media-related information about the medium displayed on the physical medium acquired from the image, and applies the media-related information to the media-related information of the electronic medium provided in the electronic payment service to generate electronic media or assist in the generation of electronic media, thereby improving convenience for users (including administrators).

[0096] The above describes the form for carrying out the present invention using an embodiment, but the present invention is not limited to such an embodiment, and various modifications and substitutions can be made within the scope that does not deviate from the gist of the present invention. [Explanation of symbols]

[0097] 10 User terminal device 20. Payment App 70 Second store terminal device 72 Merchant Interface 100 Payment Server 120 Contents Provider 150 Generation part 152 First acquisition part 154 Second Acquisition Department 156 Processing section

Claims

1. a first acquisition unit that acquires an image of a physical medium to which a special benefit is provided according to a set condition; a second acquisition unit that acquires media-related information related to the media displayed on the physical medium obtained from the image; a processing unit that applies the medium-related information to medium-related information of an electronic medium provided in an electronic payment service to generate the electronic medium or assist in generating the electronic medium; An information processing device comprising:

2. The medium is a stamp card or a point card. The information processing device according to claim 1 .

3. The medium-related information is information indicated by characters or symbols written on the physical medium. The information processing device according to claim 2 .

4. The medium-related information includes information indicating a condition for granting a privilege to a user, an expiration date of the medium, information indicating a condition for changing a degree of achievement regarding the granting of the privilege, or information regarding an area to which information indicating the degree of achievement is associated. The information processing device according to claim 3 .

5. According to instructions from the administrator of the affiliated store of the electronic payment service, the second acquisition unit acquires a combination of each of a plurality of pieces of medium-related information included in the physical medium and a category corresponding to each of the plurality of pieces of medium-related information; The processing unit, generating the electronic medium by setting each of the plurality of pieces of medium-related information acquired by the second acquisition unit to the medium-related information of the electronic medium having the common category; In response to a request from a user who uses the electronic payment service, the generated electronic medium is provided to a user terminal device of the user. The information processing device according to claim 1 .

6. According to instructions from the administrator of the affiliated store of the electronic payment service, the second acquisition unit acquires a combination of each of a plurality of pieces of medium-related information included in the physical medium and a category corresponding to each of the plurality of pieces of medium-related information; The processing unit, Presetting each of the plurality of pieces of medium-related information acquired by the second acquisition unit in a setting field for setting the medium-related information of the electronic media having the common category; generating the electronic medium in response to an operation of the administrator to provide the medium of approval or modification of the medium-related information preset in the setting field; In response to a request from a user who uses the electronic payment service, the generated electronic medium is provided to a user terminal device of the user. The information processing device according to claim 1 .

7. The processing unit includes: Inputting character information extracted from the image into a trained model, and generating the electronic medium or assisting in the generation of the electronic medium using a combination of medium-related information output by the trained model and a category of the medium-related information; The trained model is a model trained to output, when the character information is input, the medium-related information corresponding to the character information and a category of the medium-related information.

7. The information processing device according to claim 5 or 6.

8. The category of the medium-related information includes a category related to a condition for granting a privilege to a user, an expiration date of the medium, a condition for changing a degree of achievement related to the granting of the privilege, or an area to which information indicating the degree of achievement is associated.

7. The information processing device according to claim 5 or 6.

9. The medium is a stamp card to which stamps are given in response to use of a store, The second acquisition unit acquires the stamp from the image, The processing unit preferentially recommends a stamp having characteristics similar to those of the acquired stamp from among a plurality of stamps prepared in advance, or sets the stamp having characteristics similar to those of the acquired stamp as a stamp to be used on a stamp card.

3. The information processing device according to claim 1 or 2.

10. The medium is a stamp card to which stamps are given in response to use of a store, The second acquisition unit acquires a feature, which is a color or a design, of the stamp card from the image, The processing unit preferentially recommends an electronic stamp card having characteristics similar to the acquired characteristics from among a plurality of designs prepared in advance, or sets the electronic stamp card having characteristics similar to the acquired characteristics as an electronic stamp card to be used.

3. The information processing device according to claim 1 or 2.

11. In response to a request from a user who uses the electronic payment service to make an electronic payment at a store to purchase a product or service, The second acquisition unit acquires a combination of each of a plurality of pieces of medium-related information included in the physical medium and a category corresponding to each of the plurality of pieces of medium-related information; The processing unit includes: generating the electronic medium by setting each of the plurality of pieces of medium-related information acquired by the second acquisition unit to the medium-related information of the electronic medium having the common category; Correlating the generated electronic medium with the identification information of the user managed by the electronic payment service, and setting the electronic medium to a usable state in the electronic payment service.

3. The information processing device according to claim 1 or 2.

12. The processing unit acquires information indicating permission to set the electronic medium to a usable state from a terminal device of a manager of a store that provides the electronic medium, or sets the electronic medium to the usable state when the information is acquired. The information processing device according to claim 11.

13. The computer Acquire an image of a physical medium to which a special benefit is granted according to a set condition; obtaining media-related information regarding media displayed on the physical medium obtained from the image; applying the medium-related information to medium-related information of an electronic medium provided in an electronic payment service to generate the electronic medium or assist in generating the electronic medium; Information processing methods.

14. On the computer, A process of acquiring an image of a physical medium to which a special benefit is given according to a set condition; obtaining media-related information about the media displayed on the physical medium from the image; applying the medium-related information to medium-related information of an electronic medium provided in an electronic payment service to generate the electronic medium or assist in the generation of the electronic medium; A program that executes the following.

Images