Payment system

Abstract

To provide a payment system that makes a contribution to development of a bright city.SOLUTION: A payment system 1 includes a server 2, a customer database 3, and a communication terminal 4. The server 2 includes a biological authentication unit 21, an expression recognition unit 22, and a payment processing unit 23. Customer information about a plurality of customers is registered in the customer database 3. The customer information includes face information about the respective customers and payment information for conducting cashless payment for the customers. The communication terminal 4 has a camera, and is connected to the server 2. The biological authentication unit 21 performs face authentication to check the identity between a face image received from the communication terminal 4 and face information registered in the customer database 3. The expression recognition unit 22 recognizes an expression in the face image. When payment processing is performed for a customer C for whom the face authentication has been successfully performed, the payment processing unit 23 gives a prescribed merit to the customer C on the basis of a recognition result of the expression of the customer C recognized by the expression recognition unit 22.SELECTED DRAWING: Figure 1

Description

Technical Field

[0001] The present invention relates to a payment system for performing payment when a customer purchases a product or service.

Background Art

[0002] Conventionally, a system for performing payment processing using face authentication of customers visiting a store has been known (see, for example, Patent Document 1). Also, a vending machine that performs payment by face authentication of a customer attempting to purchase a product has been known (see, for example, Patent Document 2).

[0003] If payment can be made by face authentication when purchasing a product, it is convenient for customers to shop or use services without carrying cash, credit cards, etc.

[0004] However, in recent years, due to the global pandemic of infectious diseases and the instability of the international situation, the atmosphere in the world has become gloomy, which has had an adverse impact on consumer psychology. It is difficult to improve the gloomy atmosphere just by making shopping etc. convenient by face authentication.

Prior Art Documents

Patent Documents

[0005]

Patent Document 1

Patent Document 2

Summary of the Invention

Problems to be Solved by the Invention

[0006] The present invention solves the above problems and aims to provide a payment system that contributes to creating a bright town.

Means for Solving the Problems

[0007] The present invention relates to a payment system for making payments when a customer purchases goods or services, comprising a server, a customer database in which customer information of multiple customers is registered, and a communication terminal having a camera and connected to the server, wherein the server has a biometric authentication unit, a facial recognition unit, and a payment processing unit, the customer information includes the facial information of each customer and payment information for making cashless payments for that customer, the communication terminal transmits the payment amount and the customer's facial image captured by the camera to the server, the biometric authentication unit verifies the identity of the facial image received from the communication terminal with the facial information registered in the customer database by facial authentication, the facial recognition unit recognizes the facial expression in the facial image, and when the payment processing unit processes the payment for a customer whose facial authentication has been successful, it grants the customer a predetermined benefit based on the facial expression recognition result of the facial recognition unit.

[0008] In this payment system, it is preferable that the payment processing unit grants a predetermined benefit to the customer when the recognition result of the facial expression is a smile.

[0009] In this payment system, the predetermined benefit is preferably a discount on the payment amount or the awarding of points in the cashless payment.

[0010] In this payment system, the server is connected to the system of a business operator providing cashless payment services, and the payment processing unit preferably transmits the payment information, payment amount, and predetermined benefit information to the business operator's system during the payment processing.

[0011] In this payment system, each customer's cashless payment is a credit card, debit card, electronic money, or code payment, and in the payment processing of a customer who has successfully undergone facial recognition, it is preferable that the payment system does not require the customer to present a credit card, debit card, IC card for electronic money, or code for code payment.

[0012] In this payment system, the communication terminal further has a microphone, the server further has a voice recognition unit, the communication terminal further transmits the customer's voice recorded by the microphone to the server, the voice recognition unit recognizes the customer's voice, and when the payment processing unit processes the payment for a customer whose facial recognition has been successful, it may further grant the customer predetermined benefits based on the content of the customer's voice recognized by the voice recognition unit. [Effects of the Invention]

[0013] According to the payment system of the present invention, when processing a customer's payment, a predetermined benefit is granted to the customer based on the recognition result of the customer's facial expression, and can therefore be used to brighten the customer's face. If there are more customers with bright faces, it will contribute to creating a brighter city. [Brief explanation of the drawing]

[0014] [Figure 1] Figure 1 is a diagram showing the configuration of a payment system according to the first embodiment of the present invention. [Figure 2] Figure 2 is a sequence diagram of the customer information registration process in the same system. [Figure 3] Figure 3 is a sequence diagram of the payment processing in the same system. [Figure 4] Figure 4 is a sequence diagram of a modified example of the payment processing in the same system. [Figure 5] Figure 5 is a diagram showing the configuration of a payment system according to a second embodiment of the present invention. [Figure 6] Figure 6 is a sequence diagram of the payment processing in the same system. [Modes for carrying out the invention]

[0015] (First embodiment) A settlement system according to a first embodiment of the present invention will be described with reference to FIGS. 1 to 4. As shown in FIG. 1, the settlement system 1 is a system for performing settlement when a customer C purchases goods or services. The settlement system 1 includes a server 2, a customer database 3, and a communication terminal 4. The server 2 has a biometric authentication unit 21, an expression recognition unit 22, and a settlement processing unit 23. The customer database 3 stores customer information of a plurality of customers. The customer information includes face information of each customer and settlement information for performing cashless settlement for that customer. The communication terminal 4 has a camera and is connected to the server 2.

[0016] The configuration of the settlement system 1 will be further described. The server 2 is a computer and provides functions by executing programs. The biometric authentication unit 21, the expression recognition unit 22, and the settlement processing unit 23 of the server 2 are functional parts provided by executing programs respectively. The biometric authentication unit 21 performs face authentication. The expression recognition unit 22 recognizes expressions. The settlement processing unit 23 performs processing during settlement. That is, the server 2 has functions of performing face authentication, recognizing expressions, and performing processing during settlement.

[0017] Generally, authentication is to confirm whether it is the person himself or not. Face authentication is authentication using the face as a key. Face authentication confirms the identity between a collated face image and a registered image. Authentication for confirming the identity between one collated image and multiple registered images is one-to-N authentication. N in one-to-N authentication means multiple. One-to-N authentication is to repeat one-to-one authentication N times. A person whose identity between the collated image and the registered image is confirmed has succeeded in face authentication and has been identified from the registered persons. Such face authentication is performed by AI.

[0018] In the payment system 1, the face image of customer C received from the communication terminal 4 is the comparison image in face authentication. The face information registered in the customer database 3 is the information of the registered image in face authentication. A plurality of customers' information is registered in the customer database 3, and face authentication in the payment system 1 is one-to-N authentication. A person whose identity between the face image and the face information registered in the customer database 3 is confirmed is a customer who has succeeded in face authentication and has been identified from the customer database 3. Such face authentication is performed by the biometric authentication unit 21. The biometric authentication unit 21 has a face authentication algorithm using deep learning.

[0019] Note that face authentication is one type of biometric authentication. In addition to face authentication, biometric authentication includes iris authentication, voice authentication, and the like.

[0020] It is possible to recognize a person's expression by computer. For example, Japanese Industrial Standard JIS S0020 "Accessible Design - Accessibility Evaluation Method for Consumer Products" describes input by expression as one of the alternative modes of voice input. In addition, a system for recognizing expressions and confirming advertising effects is known (see, for example, Japanese Patent Laid-Open No. 2017-187927).

[0021] However, conventionally, expressions have been considered unfavorable parameters in face authentication, so they have not been used together with face authentication. According to Japanese Industrial Standard JIS X8101-2:2010 "Information Technology - Biometric Performance Testing and Reporting - Part 2: Test Methods for Technology Evaluation and Scenario Evaluation", expression is one of the accuracy influencing factors in a face authentication system. Changes in face shape and wrinkles due to expression changes affect the error rate of face authentication, and it is said that the accuracy is better when it is closer to a blank expression. However, this standard is based on the international standard ISO / IEC 19795-2:2007 in 2007, which is technology more than 10 years old. Since then, face authentication technology has advanced rapidly, and currently, there is no need to limit it to a blank expression in face authentication.

[0022] Customer database 3 is constructed using a database server. That is, both server 2 and customer database 3 are types of servers, and in this embodiment, they are implemented using cloud computing 20. Server 2 and customer database 3 may be configured as a single unit. Note that cloud computing is not required for server 2 and customer database 3.

[0023] The facial information registered in customer database 3 consists of features extracted from the customer's facial image for facial recognition. The customer payment information registered in customer database 3 is information for cashless payments made by the customer. For example, if the cashless payment is by credit card, this includes the name of the credit card holder, the credit card number, and the credit card's expiration date. The payment amount varies for each transaction and is not included in this payment information.

[0024] In this embodiment, the communication terminal 4 is a smartphone. The communication terminal 4 may also be a tablet terminal or PC with a camera.

[0025] The process for registering customer information in the payment system 1 configured as described above will now be explained. Server 2 further includes a registration processing unit 24 as its functional part. Customer C uses their customer communication terminal 6 to register customer information. In this embodiment, customer communication terminal 6 is customer C's smartphone. Note that customer communication terminal 6 may be a tablet terminal or PC with a camera.

[0026] Customer C takes a picture of their face with their smartphone and saves the image to their smartphone. Customer C accesses payment system 1 using their smartphone's browser or a dedicated app. When registering customer information, customer C's smartphone, i.e., customer communication terminal 6, connects to server 2 of payment system 1.

[0027] Figure 2 shows the main flow of the customer information registration process. Note that detailed exception handling and other design considerations are omitted from the diagram.

[0028] The customer communication terminal 6 transmits customer C's facial image and payment information to the server 2. The registration processing unit 24 of the server 2 transmits the received payment information to the cashless payment provider's system 5 and requests verification of the payment information. The cashless payment provider's system 5 verifies the received payment information to prevent fraudulent use and returns the verification result (pass / failed) to the server 2. If the verification result is failed, the registration processing unit 24 displays this to customer C's customer communication terminal 6 and stops processing; if the verification result is passed, it proceeds to the next step.

[0029] The biometric authentication unit 21 of server 2 extracts feature quantities to be used for facial recognition from customer C's facial image. The registration processing unit 24 uses these feature quantities as customer C's facial information. Feature quantities are numerical sequences that represent differences in individual faces. While it is technically possible to use the facial image directly as facial information, using feature quantities results in faster facial recognition. The registration processing unit 24 associates (links) customer C's facial information with payment information and registers it in the customer database 3. The data registered in the customer database 3 is encrypted. Server 2 then notifies customer C's customer communication terminal 6 that registration is complete.

[0030] Next, the payment processing in payment system 1 will be explained (see Figure 1). Staff S at the store use a communication terminal 4 to process payments for customer C's purchases. In this embodiment, the communication terminal 4 is a smartphone at the store. The communication terminal 4 is connected to the server 2 of payment system 1 via a network.

[0031] Figure 3 shows the main flow of the payment processing. Note that detailed exception handling and other design considerations are omitted from the diagram.

[0032] When customer C purchases goods or services, store staff S takes a picture of customer C's face with a communication terminal 4 and enters the payment amount into the communication terminal 4 using a browser or dedicated application. The communication terminal 4 transmits the payment amount and the image of customer C's face taken by the camera of the communication terminal 4 to the server 2. In this embodiment, the transmission and reception of data between the communication terminal 4 and the server 2 during the payment process is performed via the payment processing unit 23.

[0033] The biometric authentication unit 21 verifies the identity of the facial image received from the communication terminal 4 with the facial information registered in the customer database 3 through facial recognition. This facial recognition is a one-to-many authentication. The biometric authentication unit 21 passes the authentication result to the payment processing unit 23. If the authentication result is a failure, the payment processing unit 23 notifies the communication terminal 4 of the authentication failure. Upon seeing the authentication failure notification, staff member S will ask customer C to use another payment method such as cash. If the authentication result is a success, customer C, who has successfully undergone facial recognition, is identified. The payment processing unit 23 retrieves customer C's payment information from the customer database 3.

[0034] The facial expression recognition unit 22 recognizes the facial expression in the face image received from the communication terminal 4.

[0035] When the payment processing unit 23 processes a payment for customer C who has successfully undergone facial recognition, it grants a predetermined benefit to customer C based on the facial expression of customer C as recognized by the facial expression recognition unit 22. It is desirable to grant a benefit when customer C has a cheerful expression.

[0036] In this embodiment, when the facial expression recognition result is a smile, a predetermined benefit is granted to customer C.

[0037] To create a brighter atmosphere in stores and other establishments, it is desirable to inform customers (C) in advance that they will receive a reward for smiling. According to learning psychology, the provision of a reward acts as a reinforcer, thus reinforcing the behavior of smiling.

[0038] Normally, people smile when they find something funny or enjoyable, but according to one theory in neuroscience, the reverse process also exists. Human behavior and brain function are interconnected, and moving facial muscles is also a form of behavior. It is said that when you shape your facial muscles into a smile, that signal is fed back to the brain, making you feel something is funny or enjoyable.

[0039] Furthermore, if customer C smiles, others who see it will likely smile as well. According to behavioral psychology, there is a phenomenon called conformity behavior, where people tend to act in the same way as those around them, and facial expressions are also a form of behavior.

[0040] Furthermore, if a staff member S at a store or similar establishment evaluates customer C's facial expression, there is a risk that staff member S may receive complaints from customer C who gives a low evaluation. In this invention, since customer C's facial expression is automatically recognized, such a risk can be avoided. This is similar to how karaoke scoring is made more like a game when done automatically by a karaoke machine, thus avoiding any potential conflict.

[0041] As shown in Figure 4, after successful facial recognition, the camera of the communication terminal 4 may take another picture of customer C's face and send that facial image to the server 2, whereupon the facial expression recognition unit 22 may recognize the expression in the retransmitted facial image. This allows the staff S at the store or other establishment to prompt customer C, who was expressionless during facial recognition, to smile.

[0042] The predetermined benefits to be granted to customer C include, for example, a discount on the payment amount or the awarding of points in the aforementioned cashless payment. Discounts and point awards serve as incentives for shopping.

[0043] Server 2 is connected to the system 5 of the business operator providing the cashless payment service (see Figure 1). In the payment processing, the payment processing unit 23 transmits payment information, payment amount, and information on predetermined benefits to be granted (for example, discount on the payment amount or information on point granting for cashless payments) to the business operator's system 5 (see Figure 3). The business operator's system 5 executes the payment based on the information received from Server 2. Then, the business operator's system 5 notifies Server 2 that the payment is complete. Then, the payment processing unit 23 notifies the communication terminal 4 of the payment completion and benefit granting information.

[0044] Depending on the facial expression of customer C recognized by the facial recognition unit 22, it is possible that no benefit will be granted to customer C, meaning that the benefit may be 0 yen or 0 points.

[0045] The predetermined benefits to be granted to customer C may be coupons or goods. If they do not affect the payment, the payment processing unit 23 will not send information about the benefit grant to the business operator's system 5 during the payment processing.

[0046] In this embodiment, Server 2 is connected to the systems 5 of multiple businesses that provide cashless payment services (see Figure 1). Each customer's cashless payment method is a credit card, debit card, electronic money, or code payment. Note that not all of these cashless payment methods are selectable in Payment System 1. In processing the payment of Customer C who has successfully undergone facial recognition, Payment System 1 does not ask Customer C to present a credit card, debit card, IC card for electronic money, or code for code payment (see Figures 3 and 4).

[0047] As described above, the payment system 1 according to this embodiment grants a predetermined benefit to a customer based on the recognition result of the customer's facial expression when processing the customer's payment, and can therefore be used to brighten the faces of customers who purchase goods or services.

[0048] If payment system 1 brings a brighter look to the faces of customers using stores and other establishments, the atmosphere of those establishments will become brighter, which can be expected to attract more customers. A larger number of customers with brighter faces will contribute to creating a more vibrant city.

[0049] By offering a benefit when customer C is smiling, customer C will smile again.

[0050] By offering customer C a discount on the payment amount or awarding points for cashless payments, that benefit becomes an incentive for shopping and using services.

[0051] By connecting payment system 1 to the system of a business providing cashless payment services, it can support various cashless payment methods.

[0052] By not requiring customer C to present a credit card or similar, payment system 1 allows customer C to make purchases without having a credit card or similar.

[0053] (Second embodiment) A payment system 1 according to a second embodiment of the present invention will be described with reference to Figures 5 and 6. As shown in Figure 5, the payment system 1 of this embodiment has the same configuration as the first embodiment and further includes a voice recognition unit 25 for voice recognition. The same reference numerals are used for parts equivalent to those in the first embodiment. Detailed descriptions of parts equivalent to those in the first embodiment will be omitted in the following description.

[0054] The payment system 1 comprises a server 2, a customer database 3, and a communication terminal 4. The server 2 has a biometric authentication unit 21, a facial recognition unit 22, and a payment processing unit 23, and further has a voice recognition unit 25. The customer database 3 registers customer information for multiple customers. This customer information includes each customer's facial information and payment information for cashless payments for that customer. The communication terminal 4 has a camera and further has a microphone, and is connected to the server 2.

[0055] As shown in Figure 6, when customer C purchases goods or services, store staff S takes a picture of customer C's face with a communication terminal 4 and enters the payment amount into the communication terminal 4 using a browser or dedicated application. The communication terminal 4 sends the payment amount and the image of customer C's face taken by the camera of the communication terminal 4 to the server 2.

[0056] The biometric authentication unit 21 verifies the identity of the facial image received from the communication terminal 4 with the facial information registered in the customer database 3 through facial recognition. This facial recognition is a one-to-many authentication. The biometric authentication unit 21 passes the authentication result to the payment processing unit 23. If the authentication result is a failure, the payment processing unit 23 notifies the communication terminal 4 of the authentication failure. Upon seeing the authentication failure notification, staff member S will ask customer C to use another payment method such as cash. If the authentication result is a success, customer C, who has successfully undergone facial recognition, is identified. The payment processing unit 23 retrieves customer C's payment information from the customer database 3.

[0057] The facial expression recognition unit 22 recognizes the facial expression in the face image received from the communication terminal 4.

[0058] The communication terminal 4 transmits the voice (voice data) of customer C, recorded by the microphone, to the server 2. The voice recognition unit 25 recognizes the received voice of customer C.

[0059] When processing a payment for customer C who has successfully undergone facial recognition, the payment processing unit 23 grants customer C a predetermined benefit based on the facial expression of customer C recognized by the facial expression recognition unit 22 and the content of customer C's voice recognized by the voice recognition unit 25.

[0060] The audio message used when granting a benefit might be, for example, "Thank you." This allows payment system 1 to reward customer C's kind (heartfelt) words and contribute to creating a vibrant city.

[0061] The voice message used when offering a benefit could be, for example, "I don't need a plastic bag." This allows payment system 1 to contribute to creating an environmentally friendly city.

[0062] In the first and second embodiments, the payment system 1 may combine facial recognition and iris recognition. In this case, the customer information registered in the customer database 3 further includes the iris information of each customer. The facial image of customer C captured by the communication terminal 4 includes the iris image of customer C. The biometric authentication unit 21 verifies the identity of the facial image and iris image received from the communication terminal 4 with the facial information and iris information of the customer registered in the customer database 3 using facial recognition and iris recognition.

[0063] In this way, combining facial recognition and iris recognition further improves the accuracy of customer authentication. Furthermore, even if part of customer C's face is covered by a mask, the iris is not covered by the mask, allowing for highly accurate authentication.

[0064] Furthermore, the present invention is not limited to the configuration of the above embodiments, and various modifications are possible without changing the essence of the invention. For example, the communication terminal 4 is not limited to a smartphone in a store, but may be a device incorporated into a self-checkout or vending machine. Also, in the second embodiment, the payment system 1 may be configured to perform facial recognition and voice recognition. [Explanation of symbols]

[0065] 1. Payment System 2 servers 21 Biometrics Department 22 Facial expression recognition section 23 Payment Processing Unit 25. Voice Recognition Unit 3. Customer Database 4. Communication terminals 5. Systems of businesses providing cashless payment services

Claims

1. A payment system for making payments when a customer who has been informed in advance that a benefit will be given if they have a smiling expression purchases a product or service, Server and A customer database containing customer information for multiple customers, It has a camera and a communication terminal connected to the server, The aforementioned server has a biometric authentication unit, a facial recognition unit, and a payment processing unit, and is connected to the systems of multiple businesses that provide cashless payment services. The aforementioned customer information includes each customer's facial information and payment information for cashless payments made by that customer. The communication terminal transmits the payment amount and the customer's facial image captured by the camera to the server. The biometric authentication unit verifies the identity between the facial image received from the communication terminal and the facial information registered in the customer database through facial authentication. The facial expression recognition unit recognizes the facial expression in the face image, When the payment processing unit processes a payment for a customer whose facial recognition has been successful, if the facial expression recognition unit determines that the customer's facial expression is smiling, it grants the customer a predetermined benefit. The payment processing unit is characterized in that, in the payment processing, it transmits the payment information, the payment amount, and the information of the predetermined benefits to one of the business operators' systems.

2. The payment system according to claim 1, characterized in that the predetermined benefit is a discount on the payment amount or the awarding of points in the cashless payment.

3. A payment system for making payments when a customer who has been informed in advance that a benefit will be given if they have a smiling expression purchases a product or service, Server and A customer database containing customer information for multiple customers, It has a camera and a communication terminal connected to the server, The aforementioned server has a biometric authentication unit, a facial recognition unit, and a payment processing unit, and is connected to the systems of multiple businesses that provide cashless payment services. The aforementioned customer information includes each customer's facial information and payment information for cashless payments made by that customer. The communication terminal transmits the payment amount and the customer's facial image captured by the camera to the server. The biometric authentication unit verifies the identity between the facial image received from the communication terminal and the facial information registered in the customer database through facial authentication. The facial expression recognition unit recognizes the facial expression in the face image, When the payment processing unit processes a payment for a customer whose facial recognition has been successful, if the facial expression recognition unit determines that the customer's facial expression is smiling, it grants the customer a predetermined benefit. The aforementioned benefits do not affect the aforementioned cashless payment system. The payment processing unit is characterized in that, in the payment processing, it transmits the payment information and payment amount information to one of the business operators' systems.

4. Each customer's cashless payment method is credit card, debit card, electronic money, or QR code payment. The payment system according to claim 1 or 3, characterized in that, in processing the payment of a customer whose facial recognition has been successful, the payment system does not require the customer to present a credit card, debit card, IC card for electronic money, or code for code payment.

5. The aforementioned communication terminal further includes a microphone, The server further includes a speech recognition unit, The communication terminal further transmits the customer's voice, recorded by the microphone, to the server. The voice recognition unit recognizes the customer's voice, The payment system according to claim 1 or 3, characterized in that when the payment processing unit processes a payment for a customer whose facial recognition has been successful, it further grants the customer a predetermined benefit based on the content of the customer's voice recognized by the voice recognition unit.

Images