IDENTIFYING CONSUMERS IN A TRANSACTION USING FACIAL RECOGNITION

Abstract

A computer-implemented method for identifying users at locations by comparing facial images of users (101) with facial templates of users (101), comprising: Transmitting an identifier by a computing device that is associated with the computing device at a location; Receiving, by the computing device and from one or more other computing devices, one or more facial templates, each facial template associated with a corresponding user computing device (110) that received the transmitted identifier at the location and further transmitted the identifier to the one or more computing devices, each facial template comprising a representation of the corresponding facial image of the corresponding user; Capturing, by a camera module of the computing device, a video feed of an environment external to the computing device; extracting, by the computing device, a facial image of a user (101) from the video feed; generating, by the computing device, a facial template of the user (101) based on the extracted facial image of the user (101), wherein the facial template of the user (101) is a representation of the facial image of the user (101); Retrieving, by the computing device, one or more stored facial templates; determining, by the computing device, a similarity between the generated facial template of the user (101) and each of the one or more stored facial templates; identifying, by the computing device, a particular stored face template corresponding to the generated face template of the user (101) based on the similarity between the generated face template of the user (101) and the particular stored face template exceeding a threshold; Sending, by the computing device, a response to a request to identify a user (101) comprising an indication of the identified user (101).

Description

TECHNICAL FIELD

[0001] The present disclosure relates to improving the user experience in transactions by identifying user accounts for use in transactions based on facial recognition of the users by a payment processing system. GENERAL STATE OF THE ART

[0002] When consumers make purchases at a merchant location, many methods are available for completing a transaction. Consumers can use many different cards or accounts to make purchases, such as gift cards, store cards, credit cards, debit cards, and other cards or accounts. User account identifiers and other data represented by the cards can be communicated to the merchant system via the cards' magnetic stripes, near-field communication technologies involving user computing devices, and other suitable mechanisms.

[0003] Current applications for conducting transactions at a merchant location do not provide the consumer with a way to conduct a hands-free transaction. Furthermore, current applications require the consumer to perform actions to identify themselves by providing user account identifiers or other data to the merchant system.

[0004] US 2015 / 0 073 907 A1 relates to wearable devices, methods and systems for intelligent vision devices (“WIVD”). SUMMARY OF REVELATION

[0005] It is disclosed in a computer-implemented method according to claims 1 to 6, a computer-implemented method according to claims 7 to 11, a computer program product according to claims 12 to 16 and a system according to claims 17 to 20.

[0006] Techniques herein provide computer-implemented methods for conducting a hands-free transaction with facial recognition of a user. In an exemplary embodiment, a merchant logs into a payment processing system. A user establishes an account with the payment processing system and sends a picture of themselves to the payment processing system to establish a facial template associated with the user account. The user logs into the payment application via the user computing device and enters the merchant system location. The user computing device receives a merchant beacon device identifier from the merchant beacon device and sends the identifier to the payment processing system. The payment processing system sends facial templates to the merchant camera device corresponding to other users whose user computing devices are within network range of the merchant beacon device and who are logged into the payment application.The merchant camera device captures a facial image of the user and identifies the user by comparing the captured facial image with the received facial templates. The merchant point-of-sale device operator selects a user account for use in a transaction from one or more displayed user accounts. The merchant point-of-sale device sends transaction details to the payment processing system, which generates a transaction approval request that is sent to an issuer system associated with the user account selected for use in the transaction. The payment processing system receives approval of the transaction approval request and sends a receipt acknowledgment to the merchant point-of-sale device.

[0007] In certain other exemplary aspects described herein, systems and computer program products are provided for conducting a hands-free transaction with facial recognition of a user.

[0008] These and other aspects, objects, features, and advantages of the exemplary embodiment will be apparent to those skilled in the art after studying the following detailed description of illustrated exemplary embodiments. BRIEF DESCRIPTION OF THE DRAWINGS Fig. 1 is a block diagram illustrating a system for performing a hands-free transaction with facial recognition of a user in accordance with certain example embodiments. Fig. 2 is a block flow diagram illustrating a method for performing a hands-free transaction with facial recognition of a user according to certain example embodiments. Fig. 3 is a block flow diagram illustrating a method for logging into a payment processing system by a merchant system and installing hardware at a merchant system site, according to certain example embodiments. Fig. 4 is a block flow diagram illustrating a method for signing up, by a user, for an account with a payment processing system according to certain example embodiments. Fig. 5 is a block flow diagram illustrating a method for setting up a face template associated with a user account, according to certain example embodiments. Fig. 6 is a block flow diagram illustrating a method for setting up an audio template associated with a user account, according to certain example embodiments. Fig. 7 is a block flow diagram illustrating a method for receiving a merchant beacon identifier transmitted by a merchant beacon device by a user computing device, according to certain example embodiments. Fig. 8 is a block flow diagram illustrating a method for receiving a facial template for each user in range of the merchant beacon device by a camera device, according to certain example embodiments. Fig. 9 is a block flow diagram illustrating a method for receiving notification from a payment processing system through a camera device when users enter or exit a network area of a merchant beacon device, according to certain example embodiments. Fig. 10 is a block flow diagram illustrating a method for initiating a transaction at a merchant point of sale device by a user, according to certain example embodiments. Fig. 11 is a block flow diagram illustrating a method for identifying a user via facial recognition by a camera device according to certain example embodiments. Fig. 12 is a block flow diagram illustrating a method for logging into a processing system by a user via speech recognition, according to certain example embodiments. Fig. 13 is a block flow diagram illustrating a method for identifying a user via a question and response by a merchant point-of-sale device operator according to certain example embodiments. Fig. 14 is a block flow diagram illustrating a method for performing a transaction according to certain example embodiments. Fig. 15 is a block diagram illustrating a computing engine and module according to certain example embodiments. DETAILED DESCRIPTION OF THE EXEMPLARY EMBODIMENTSOverview

[0009] The exemplary embodiments described herein provide computer-implemented techniques for performing a hands-free transaction with facial recognition of a user.

[0010] In an exemplary embodiment, a merchant logs into a payment processing system. A merchant system installs one or more merchant beacon devices, one or more merchant point-of-sale devices, and one or more merchant camera devices at a merchant system location. For example, a user sets up an account with the payment processing system and downloads a payment application to a user computing device associated with the user. In one example, the user sends a picture of themselves and / or an audio recording of themselves to the payment processing system to set up a face template and / or an audio template associated with the user account. A user enters a merchant system location and logs into the payment application via the user computing device. The user computing device receives a merchant beacon device identifier from the merchant beacon device and sends the identifier to the payment processing system.The payment processing system sends facial templates to the merchant camera device corresponding to users whose user computing devices are within network range of the merchant beacon device and who are logged into the payment application. The merchant camera device captures a facial image of the user and identifies the user based on comparing the captured facial image with the received facial templates. Alternatively, the user sends an audio recording via the merchant point-of-sale device to the payment processing system, which identifies the user based on comparing the received audio recording with user audio templates. In yet another embodiment, the merchant point-of-sale device operator identifies the user based on a user's response to a question. After identifying the user, the merchant point-of-sale device displays one or more of the user's accounts.The merchant point-of-sale device operator selects a user account for use in a transaction. The merchant point-of-sale device sends transaction details to the payment processing system, which generates a transaction approval request, which is sent to an issuer system associated with the user account selected for use in the transaction. The payment processing system receives approval of the transaction approval request and sends a receipt to the merchant point-of-sale device.

[0011] In one exemplary embodiment, a merchant system registers with a payment processing system. A merchant system operator installs a payment application on a merchant point-of-sale device. In another example, the merchant system operator installs the payment application on multiple merchant point-of-sale devices at a merchant system location. A merchant beacon device receives a beacon identifier from a payment processing system. For example, the merchant system operator installs one or more merchant beacon devices at the merchant system location. The merchant beacon device transmits the merchant beacon identifier via wireless communication at the merchant system location. The merchant system operator installs a merchant camera device at the merchant system location to correspond to the merchant point-of-sale device.In another example, a plurality of merchant camera devices are installed at the merchant system site, each merchant camera device corresponding to a particular merchant point-of-sale device. In yet another example, a particular merchant camera device may correspond to two or more particular merchant point-of-sale devices. The payment processing system receives a merchant camera device identifier and associates it with the corresponding beacon identifier of the merchant beacon device.

[0012] In an exemplary embodiment, the user logs into a payment processing system. For example, the user accesses a payment processing system website via a user computing device connected to the user. The user logs into the payment processing system and downloads a payment application to the user computing device. In an exemplary embodiment, the payment processing system sets up a facial template associated with the user account. For example, the payment application displays a request to the user to capture a facial image via the user computing device. The user selects an option to capture a facial image. The payment application activates a camera module on the user computing device, and the user captures a facial image of themselves. The payment processing system receives the facial image.The payment processing system creates a face template associated with the user account based on the received facial image. The payment processing system deletes the received facial image. In another exemplary embodiment, the payment processing system establishes an audio template associated with the user account. The payment application requests the user audio and receives it via the user computing device. The payment application creates an audio template associated with the user account based on the received audio of the user's voice. The payment processing system deletes the received audio of the user's voice.

[0013] The user logs into a payment application on the user computing device. The user carries the user computing device within a threshold distance of a merchant beacon device at the merchant system location. The user computing device receives a merchant beacon identifier transmitted from the merchant beacon device and sends the received merchant beacon identifier and a user account identifier to the payment processing system. The payment processing system receives the merchant beacon identifier and the user account identifier. The payment processing system extracts a facial template associated with the user account identifier and identifies a merchant camera device associated with the merchant beacon device identifier.

[0014] The payment processing system sends a facial template of the identified user to the merchant camera device associated with the merchant beacon device identifier. For example, a facial template associated with the identified customer's account is sent to the merchant camera device. The merchant camera device receives the user's facial template. The merchant camera device adds the user's facial template to the current customer log. The merchant camera device updates the current customer log based on updates received from the payment processing system. For example, the payment processing system sends a subsequent facial template of a subsequent user who, carrying a user computing device through which the user logged into the payment application, enters a threshold distance of a merchant beacon device required to establish a wireless network connection.In this example, the payment processing system receives the merchant beacon device identifier sent by the user computing device and sends a facial template of the subsequent user to the merchant camera device. In another example, in response to detecting that the user computing device associated with a user in the current customer log is no longer maintaining a network connection with the merchant beacon device, is no longer paying attention to the merchant beacon device, or is no longer logged into the payment application, the payment processing system sends an indication that the user has left a merchant location to the merchant camera device. In this example, the merchant camera device deletes the specified user from the current customer log.

[0015] The user approaches a merchant point-of-sale device. The merchant point-of-sale device operator determines the total amount of the user's items for purchase. The merchant point-of-sale device operator asks the user to select a payment option. The user instructs the merchant point-of-sale device operator to initiate a transaction via the payment application. For example, as previously described, the payment application is installed on both the merchant point-of-sale device and the user's computing device. The merchant point-of-sale device operator selects an option at the merchant point-of-sale device to initiate a transaction using the payment application. The merchant point-of-sale device sends a request to identify the user to the merchant camera device via the payment processing system.In this example, the payment processing system communicates with the merchant camera device and the merchant point of sale device.

[0016] In another exemplary embodiment, the merchant point-of-sale device receives an indication from the merchant camera device that the user has been identified, and the merchant point-of-sale device presents the user with an option to initiate a transaction using the payment application. For example, the merchant point-of-sale device may receive an indication that the user has been identified directly from the merchant camera device over a network. In another example, the merchant camera device sends an indication that the user has been identified to the payment processing system, and the payment processing system then sends the indication that the user has been identified to the merchant point-of-sale device.In these exemplary embodiments, in response to the merchant point-of-sale device displaying an option to initiate a transaction using the payment application, the merchant point-of-sale device operator asks the user whether they wish to initiate a transaction using the payment application. In this exemplary embodiment, the user instructs the merchant point-of-sale device operator to initiate the transaction via the payment application, and the merchant point-of-sale device operator selects the appropriate option on the user interface of the merchant point-of-sale device.

[0017] The merchant camera device records video of the user. For example, the user is located in front of the point-of-sale device, and the merchant camera device is positioned to capture video of the user's face. In one exemplary embodiment, the merchant camera device starts recording video of the user only when the camera device receives a request to identify the user. For example, the merchant camera device receives a request to identify the user from the payment processing system or from the merchant point-of-sale device. In another exemplary embodiment, the merchant camera device begins recording video when the merchant camera device receives an indication from the payment processing system that a user computing device associated with the user has established a network connection to the merchant beacon device.In this exemplary embodiment, the merchant camera device does not record video if no users with connected user computing devices are within network range of the merchant beacon device. The merchant camera device extracts a facial image of the user from the recorded video and generates a facial template from the recorded facial image. The merchant camera device deletes the recorded video and the extracted facial image. The merchant camera device retrieves facial templates from the current customer log. For example, the current customer log includes a list of users and connected facial templates for users connected to user computing devices that currently have a network connection established with a merchant beacon device at the merchant system site. In one exemplary embodiment, the current customer log includes volatile or temporary memory.For example, the current customer log is not saved, and user information is added to or deleted from the current customer log when user computing devices associated with respective users enter or leave a network range of the merchant beacon device. The merchant camera device compares the generated facial template from the extracted facial image with facial templates from the current customer log. The merchant camera device can identify the user if there is a match between a facial template from the current customer log and the generated facial template. The merchant camera device cannot identify the user if there is no match between a facial template from the current customer log and the generated facial template.If the merchant camera device can identify the user, the merchant camera device notifies the payment processing system of the user's identity. In another exemplary embodiment, if the merchant camera device can identify the user, the merchant camera device notifies the merchant point-of-sale device of the user's identity.

[0018] If the merchant camera device cannot identify the user based on facial recognition, the merchant camera device notifies the payment processing system that the user cannot be identified based on facial recognition. In one exemplary embodiment, if the user cannot be identified based on facial recognition, the payment processing system identifies the user based on audio recognition. In another exemplary embodiment, the payment processing system does not identify users based on audio recognition. In one exemplary embodiment, if payment processing identifies users based on audio recognition, the payment processing system retrieves audio templates corresponding to users from the current customer log. The payment processing system sends a request to the merchant system point-of-sale device to record audio of the user.The merchant system point-of-sale device displays a request for the user to record audio, records a voice input from the user, and sends the voice input to the payment processing system. The payment processing system compares the received voice input with the retrieved audio templates from the current customer log that correspond to the users. The payment processing system can identify the user if there is a match between an audio template from the current customer log and the received voice input from the user. The payment processing system cannot identify the user if there is no match between an audio template from the current customer log and the received voice input from the user.

[0019] If the payment processing system cannot identify the user based on speech recognition, the payment processing system notifies the merchant point-of-sale device operator to issue a question to the user. The user provides an answer to the question, and the merchant point-of-sale operator enters the answer into the merchant point-of-sale device. The merchant point-of-sale device displays potential users based on the answer to the question. For example, the merchant point-of-sale device sends the answer to the payment processing system, and the payment processing system accesses a database that includes a list or table that matches questions to corresponding answers and users. In this example, the payment processing system identifies the user by correlating the question and answer to identify one or more users in the database.In this example, the payment processing system sends the one or more identified users to the merchant point-of-sale device. In this example, the merchant point-of-sale device displays the one or more identified users to the merchant point-of-sale device operator. The merchant point-of-sale device operator selects a user. In one example, a merchant point-of-sale device operator can compare a visual image or name of the user displayed on the user computing device with the visual appearance of the customer at the merchant point-of-sale device and / or on documentation presented by the user to the merchant point-of-sale device operator. In one example, the merchant point-of-sale device sends the identity of the user identified by the merchant point-of-sale operator.

[0020] The point-of-sale device displays accounts of the identified user. After receiving the user identity from the merchant camera device, receiving the user identity from the merchant point-of-sale device, or identifying the user, for example, the payment processing system sends information associated with one or more of the identified user's accounts. The merchant point-of-sale device operator selects a user account for the transaction and confirms the transaction with the user's approval. The merchant point-of-sale device sends transaction details to the payment processing system. The transaction details include, for example, a total transaction amount, a selected user account for use in the transaction, a merchant account for use in the transaction, and other useful or relevant information.The payment processing system sends a transaction approval request to an issuer system. The issuer system is associated, for example, with the user account selected for use in the transaction. The issuer system approves the transaction approval request and sends a transaction authorization approval to the payment processing system. The payment processing system sends a transaction receipt to the merchant point-of-sale device.

[0021] By using and relying on the methods and systems described herein, the payment processing system, the merchant camera device, the merchant beacon device, the user computing device, and the merchant point-of-sale device enable the user to conduct a transaction with the merchant system without requiring the user to interact with the user computing device or produce identity documents or physical payment cards, as required by some current technologies. As such, the systems and methods described herein can reduce the inputs required by the user via the user computing device and the inputs required by the merchant point-of-sale device operator to identify the user. Example system architecture

[0022] Exemplary embodiments will be described in detail with reference to the drawings, in which like numerals indicate like (but not necessarily identical) elements throughout the figures.

[0023] Fig. 1 is a block diagram 100 illustrating a system for performing a hands-free transaction with facial recognition of a user 101 according to certain exemplary embodiments. As shown in Fig. 1, system 100 includes network computing devices 110, 130, 140, 150, and 160 configured to communicate with each other over one or more networks 120. In some embodiments, a user associated with a device must install an application and / or make a feature selection to obtain the benefits of the techniques described herein.

[0024] In exemplary embodiments, network 120 may include a local area network (''LAN''), a wide area network (''WAN''), an intranet, an internet, a storage area network (''SAN''), a personal radio network (''PAN''), a metropolitan area network (''MAN''), a wireless local area network (''WLAN''), a virtual private network (''VPN''), a cellular or other mobile communications network, Bluetooth, Bluetooth Low Energy (''BLE''), near field communication (''NFC''), or any combination thereof, or any other suitable architecture or system that enables the communication of signals, data, and / or messages. Throughout the discussion of the exemplary embodiments, it should be understood that the terms "data" and "information" are used interchangeably herein to refer to text, images, audio, video, or any other form of information that may exist in a computer-based environment.

[0025] Each network computing device 110, 130, 140, 150, and 160 includes a device having a communications module capable of sending and receiving data over the network 120. Each network computing device 110, 130, 140, 150, and 160 may include, for example, a server, desktop computer, laptop, tablet computer, a television with one or more processors embedded and / or coupled thereto, a smartphone, handheld computer, personal digital assistant (''PDA''), or any other wired or wireless processor-controlled device. Fig. 1, the network computing devices 110, 130, 140, 150, and 160 are operated by users 101, merchant beacon device 120 operators, merchant point of sale (''POS'') device 130 operators, merchant camera device 140 operators, issuer system 150 operators, and payment processing system 160, respectively.

[0026] An exemplary user computing device 110 includes an antenna 111, a Wi-Fi controller 112, a payment application 113, a user interface 115, a data storage unit 116, a camera module 117, a web browser 118, and a communication application 119.

[0027] In one exemplary embodiment, antenna 111 is a means of communication between user computing device 110 and a merchant beacon device 120. In one exemplary embodiment, a Wi-Fi controller 112 transmits a radio signal through antenna 111 or listens for radio signals from merchant beacon device 120. In another exemplary embodiment, a Bluetooth controller or a near-field communication (NFC) controller is used. In one exemplary embodiment, Wi-Fi controller 112 transmits a radio signal through antenna 111 or listens for radio signals from payment card device 120.

[0028] In one exemplary embodiment, the Wi-Fi controller 112 is capable of sending and receiving data, performing authentication and encryption functions, and directing how the user computing device 110 listens for transmissions from the merchant beacon device 120, or configuring the user computing device 110 in various power-saving modes according to Wi-Fi-specified procedures. In another exemplary embodiment, the user computing device 110 includes a Bluetooth controller or an NFC controller capable of performing similar functions. An exemplary Wi-Fi controller 112 communicates with the payment application 113 and is capable of sending and receiving data over a wireless Wi-Fi communication channel. In another exemplary embodiment, a Bluetooth controller 112 or NFC controller 112 performs similar functions to the Wi-Fi controller 112 using a Bluetooth or NFC protocol.In an exemplary embodiment, Wi-Fi controller 112 activates antenna 111 to create a wireless communication channel between user computing device 110 and merchant beacon device 120. User computing device 110 communicates with merchant beacon device 120 via antenna 111. In an exemplary embodiment, once user computing device 110 has been activated, Wi-Fi controller 112 receives a radio signal or listens for radio signals from merchant beacon device 120 through antenna 111.

[0029] In an exemplary embodiment, payment application 113 is a program, function, routine, gadget, or similar entity that exists on and performs its operations on user computing device 110. In certain exemplary embodiments, user 101 must install payment application 113 and / or make feature selections on user computing device 110 to obtain the benefits of the techniques described herein. In an exemplary embodiment, user 101 may access payment application 113 on user computing device 110 via user interface 115. In an exemplary embodiment, payment application 113 may be connected to payment processing system 160.In another exemplary embodiment, the payment application 113 may be connected to a merchant system that is connected to the merchant beacon device 120, the merchant point-of-sale device 130, and the merchant camera device 140. In yet another exemplary embodiment, there are two payment applications 113, one connected to the merchant system and another connected to the payment processing system 160.

[0030]

[0043] In one exemplary embodiment, user interface 115 enables user 101 to interact with payment application 113 and / or web browser 118. User interface 115 may, for example, be a touchscreen, a voice-based interface, or any other interface that enables user 101 to provide input and receive output from an application or module on user computing device 110. In one exemplary embodiment, user 101 interacts with payment application 113 and / or web browser 118 via user interface 115 to configure user 101 accounts on payment processing system 160. In another exemplary embodiment, user 101 interacts with payment application 113 and / or web browser 118 via user interface 115 to enable hands-free payments, if necessary.

[0031] In an exemplary embodiment, data storage unit 116 includes a local or remote data storage structure suitable for storing information that is accessible to user computing device 110. In an exemplary embodiment, data storage unit 116 stores encrypted information, such as local HTML5 storage.

[0032] In an exemplary embodiment, the camera module 117 may be any module or function of the user computing device 110 that obtains a digital image. The camera module 117 may be resident on the user computing device 110 or logically connected to the user computing device 110 in some way. For example, the camera module 117 may be connected to the user computing device 110 via the network 120. The camera module 117 may be capable of obtaining individual images or a video scan. Any other suitable image capture device may be represented by the camera module 117.

[0033] In an exemplary embodiment, the user 101 may use a communications application 119, such as a web browser 118 application or a standalone application, to view, download, upload, or otherwise access documents or web pages over a distributed network 120.

[0034] In one exemplary embodiment, web browser 118 may enable user 101 to interact with web pages using user computing device 110. In one exemplary embodiment, user 101 may access user 101's account maintained by payment processing system 160 via web browser 118. In another exemplary embodiment, user 101 may access a merchant system website via web browser 118. In certain exemplary embodiments described herein, one or more functions performed by payment application 113 may also be performed by a web browser 118 application connected to payment processing system 160.

[0035] In an exemplary embodiment, the communication application 119 may interact with web servers or other computing devices connected to the network 120, including the user computing device 110 and a web server of a merchant system.

[0036] In certain exemplary embodiments, one or more functions described herein as being performed by the payment application 113 may also be performed by a web browser 118 application, such as a web browser 118 application connected to a merchant system website or connected to the payment processing system 160. In certain exemplary embodiments, one or more functions described herein as being performed by the payment application 113 may also be performed by the user computing device 110 operating system. In certain exemplary embodiments, one or more functions described herein as being performed via the web browser 118 may also be performed via the payment application 113.

[0037] An exemplary merchant beacon device 120 includes an antenna 121 and a Wi-Fi controller 122. In an exemplary embodiment, a merchant system location includes one or more merchant beacon devices 120 installed at the merchant system location. In an exemplary embodiment, each installed merchant beacon device 120 is connected by a payment processing system 160 to a particular merchant camera device 140 installed at the merchant location. The payment processing system 160 may, for example, include a database correlating the merchant beacon device 120 identifiers with the merchant camera device 140 identifiers for connected merchant camera devices 140. A merchant camera device 140 identifier may, for example, include a device-specific hardware identifier such as a serial number or a MAC ID.In another example, a merchant beacon device 120 identifier may include a hardware identifier specific to a beacon device or an identifier generated by the payment processing system 160 and stored in the merchant beacon device 120. An exemplary merchant beacon device 120 is programmed to transmit, emit, or otherwise send a particular merchant beacon device 120 identifier over a wireless network 120 to any user computing devices 110 within a threshold distance required to maintain the wireless network 120. The wireless network may include, for example, a Wi-Fi network 120, a Bluetooth network 120, an NFC network 120, or any other suitable wireless network 120.

[0038] In one exemplary embodiment, antenna 121 is a means of communication between user computing device 110 and a merchant beacon device 120. In one exemplary embodiment, a Wi-Fi controller 122 outputs a radio signal through antenna 121 or listens for radio signals from user computing device 110. In another exemplary embodiment, a Bluetooth controller or a near-field communication (NFC) controller is used. In one exemplary embodiment, Wi-Fi controller 122 outputs a radio signal through antenna 121 or listens for radio signals from user computing device 120.

[0039] In one exemplary embodiment, the Wi-Fi controller 122 is capable of sending and receiving data, performing authentication and encryption functions, and directing how the merchant beacon device 120 listens for transmissions from the user computing device 110, or configuring the merchant beacon device 120 in various power-saving modes according to Wi-Fi-specified procedures. In another exemplary embodiment, the merchant beacon device 120 includes a Bluetooth controller or an NFC controller capable of performing similar functions. An exemplary Wi-Fi controller 122 communicates with the payment application 113 and is capable of sending and receiving data over a wireless Wi-Fi communication channel. In another exemplary embodiment, a Bluetooth controller 122 or NFC controller 122 performs similar functions to the Wi-Fi controller 122 using a Bluetooth or NFC protocol.In an exemplary embodiment, Wi-Fi controller 122 activates antenna 121 to create a wireless communication channel between user computing device 110 and merchant beacon device 120. Merchant beacon device 120 communicates with user computing device 110 via antenna 121. In an exemplary embodiment, once merchant beacon device 120 has been activated, Wi-Fi controller 122 receives a radio signal or listens for radio signals from user computing device 110 through antenna 121.

[0040] An exemplary merchant point of sale device 130 includes an audio module 131, a payment application 133, a user interface 135, a data storage unit 136, and a communications application 139.

[0041] In an exemplary embodiment, the audio module 131 may be any module or function of the merchant POS device 130 that receives audio input from an environment external to the merchant POS device 130. The audio module 131 may be resident on the merchant POS device 130 or logically connected to the merchant POS device 130 in some way. For example, the audio module 131 may be connected to the merchant POS device 130 via the network 120. The audio module 131 may be capable of receiving an audio recording. Any suitable audio recording device may be represented by the audio module 131.

[0042] In an exemplary embodiment, the payment application 133 is a program, function, routine, gadget, or similar entity that exists on the merchant point-of-sale device 130 and performs its operations thereon. In certain exemplary embodiments, the merchant point-of-sale device (POS) operator 102 or other merchant system operator must install the payment application 133 and / or make a feature selection on the merchant point-of-sale device 130 to obtain the benefits of the techniques described herein. In an exemplary embodiment, the merchant POS device operator 102 may access the payment application 133 on the merchant POS device 130 via the user interface 135. In an exemplary embodiment, the payment application 133 may be connected to the payment processing system 160.In another exemplary embodiment, the payment application 133 may be connected to a merchant system that is connected to the merchant beacon device 120 and the merchant camera device 140. In yet another exemplary embodiment, there are two payment applications 133, one connected to the merchant system and another connected to the payment processing system 160.

[0043] In an exemplary embodiment, user interface 135 enables merchant POS device operator 102 to interact with merchant POS device 130. User interface 135 may, for example, be a touchscreen, a voice-based interface, or any other interface that enables merchant POS device operator 102 to provide input and receive output from an application on merchant POS device 130. In one embodiment, merchant POS device operator 102 interacts with payment application 133 via user interface 135.

[0044] In an exemplary embodiment, data storage unit 136 includes a local or remote data storage structure accessible to merchant POS device 130 and suitable for storing information. In an exemplary embodiment, data storage unit 136 stores encrypted information, such as local HTML5 storage.

[0045] In an exemplary embodiment, the communication application 139, such as a web browser application or a standalone application, enables an operator of the merchant POS device 130 to view, download, upload, or otherwise access documents or web pages over a distributed network 120. For example, the communication application 139 may enable communication over the network 120 with the payment processing system 160.

[0046] An exemplary dealer camera device 140 includes a processor 143, a data storage unit 146, a camera module 147, and a communications application 149.

[0047] In an exemplary embodiment, the processor 143 performs one or more functions described herein as being performed by the dealer camera device 140.

[0048] In an exemplary embodiment, the data storage unit 146 includes a local or remote data storage structure accessible to the merchant camera device 140 and suitable for storing information. In an exemplary embodiment, the data storage unit 146 stores encrypted information, such as local HTML5 storage.

[0049] In an exemplary embodiment, the camera module 147 may be any module or function of the dealer camera device 140 that captures a video input or a digital image of an external environment of the dealer camera device 140. The camera module 147 may be resident on the dealer camera device 140 or logically connected to the dealer camera device 140 in some way. For example, the camera module 147 may be connected to the dealer camera device 140 via the network 120. The camera module 147 may be capable of obtaining individual images or a video scan. Any other suitable image capture device may be represented by the camera module 147.

[0050] In an exemplary embodiment, the communication application 149 enables the merchant camera device 140 to communicate with the payment processing system 160.

[0051] An exemplary issuer system 150 approves or denies a payment authorization request received from payment processing system 160. In an exemplary embodiment, issuer system 150 communicates with payment processing system 160 via network 120. In an exemplary embodiment, issuer system 150 communicates with an acquirer system to approve credit authorization and make a payment to payment processing system 160 and / or merchant system. The acquirer system is, for example, a third-party payment processing company.

[0052] For example, an exemplary payment processing system 160 includes an account management module 161, a facial recognition module 163, an audio recognition module 165, a data storage unit 166, and a transaction processing module 167.

[0053] In an exemplary embodiment, the account management module 161 manages one or more user 101 accounts. In an exemplary embodiment, a user 101 account may include a digital wallet account, an email account, a social networking account, or any other suitable account connected to the payment processing system 160. In an exemplary embodiment, the account management system 161 communicates with the payment processing system 160 using a payment application 113 running on a user computing device 110 associated with a user 101 having a user 101 account. In an exemplary embodiment, the user 101 enters payment account information into the user 101 account via the payment application 113, and the account management module 161 receives the payment account information via the network 120 and associates the received payment account information with the user 101 account.

[0054] In an exemplary embodiment, facial recognition module 163 receives a facial image of a user 101 associated with a user 101 account, which was sent by user 101 via user computing device 110 over network 120. For example, user 101 sends the facial image at the time user 101 sets up user 101's account with payment processing system 160. In one embodiment, facial recognition module 163 generates a facial template based on a received facial image.

[0055] In an exemplary embodiment, the audio recognition module 165 receives an audio recording of a user 101 associated with a user 101 account, which was sent by the user 101 via the user computing device 110 over the network 120. For example, the user 101 sends the audio recording at the time the user 101 sets up the user 101 account with the payment processing system 160. In one embodiment, the audio recognition module 165 generates an audio template based on the received audio recording of the user 101. In an exemplary embodiment, the audio recognition module 165 receives an audio recording of a first user 101 and compares the audio recording of the first user 101 to a stored audio template associated with a second user 101.

[0056] In an exemplary embodiment, data storage unit 166 comprises a local or remote data storage structure accessible to payment processing system 160 and suitable for storing information. In an exemplary embodiment, data storage unit 166 stores encrypted information, such as local HTML5 storage.

[0057] In an exemplary embodiment, the transaction processing module 167 receives transaction details from a merchant POS device 130 and a request to initiate a transaction. Example transaction details include merchant system account information, a total transaction amount, and a user 101 selection of a user 101 payment account associated with the user 101 account with the payment processing system 160. The user 101 account is, for example, a digital wallet account that includes one or more payment account information corresponding to one or more corresponding payment accounts of the user 101. In an exemplary embodiment, the transaction processing module 167 extracts payment account information from the user 101 account corresponding to the user 101 selection of the user 101 payment account received in the transaction details from the merchant POS device 130.In an exemplary embodiment, transaction processing module 167 sends a payment authorization request to an issuer system 150 or other suitable financial institution associated with the payment account selected by user 101 for use in the transaction. An exemplary payment authorization request may include merchant system payment account information, user 101 payment account information, and a total transaction amount. In an exemplary embodiment, after issuer system 150 processes the payment authorization request, transaction processing module 167 receives an approval or rejection of the payment authorization request from issuer system 150 via network 120.In an exemplary embodiment, the transaction processing module 167 sends a receipt including a summary of the transaction to the merchant POS device 130 and / or the user computing device 110.

[0058] It is obvious that the network connections shown are examples, and other means for establishing a communication connection between the computers and devices may be used. Furthermore, those skilled in the art will recognize from the present disclosure that the Fig. 1, the merchant beacon device 120, the merchant point-of-sale device 130, the merchant camera device 140, the issuer system 150, and the payment processing system 160 may comprise any of several other suitable computer system configurations. For example, a user computing device 110 embodied as a mobile phone or handheld computer may or may not include all of the components described above.

[0059] In exemplary embodiments, the network computing devices and any other computing machines associated with the technology presented herein may be any type of computing machine, such as, but not limited to, those described in relation to Fig. 15 described in more detail. Furthermore, any modules associated with any of these computing engines, such as modules described herein or any other modules (scripts, web content, software, firmware, or hardware) associated with the technology presented herein may implement any of the Fig. 15. The computing machines described herein may communicate with each other and with other computing machines or communication systems via one or more networks, such as network 120. Network 120 may be any type of data or communication network, including any of the modules described with respect to Fig. 15 described network technologies. Example processes

[0060] The Fig. 2 to Fig. 14 are described below with respect to the components of the exemplary operating environment 100. The exemplary methods of Fig. 2 to Fig. 7 can also be run on other systems and in other environments.

[0061] Fig. 2 is a block diagram illustrating a system 200 for performing a hands-free transaction with facial recognition of a user 101 according to exemplary embodiments. The method 200 is described with reference to the Fig. 1 illustrated components.

[0062] At block 210, the merchant system registers with the payment processing system 160 and installs hardware at a merchant site. The method of registering, by a merchant system, with a payment processing system 160 and installing the hardware at a merchant system site is described below with reference to the Fig. 3 described procedures are described in more detail.

[0063] Fig. 3 is a block diagram illustrating a method 210 for logging on, by a merchant system, to a payment processing system 160 and installing the hardware at a merchant system site, according to certain exemplary embodiments. The method 210 is described with reference to the Fig. 1 illustrated components.

[0064] In the exemplary embodiments described herein, the merchant system at the exemplary merchant system site is not required to install hardware in any particular order. Method 210 describes an exemplary method for installing hardware at the merchant site. However, the merchant system or other system installing the merchant hardware is not required to install the merchant POS device 130, the merchant camera device 140, or the merchant beacon device 120 in the order described herein.

[0065] At block 310, a merchant system logs on to payment processing system 160. In an exemplary embodiment, an agent of the merchant system accesses a payment processing system 160 web page and signs up for a merchant account with payment processing system 160 via the web page. In an exemplary embodiment, the merchant system adds payment account information to the merchant account managed by payment processing system 160. In an exemplary embodiment, the merchant system includes one or more merchant system locations. For example, the merchant system may include one or more physical storage locations. An exemplary merchant location includes one or more merchant point-of-sale (POS) devices 130. In an exemplary embodiment, one or more merchant POS device operators 102 operate the one or more merchant POS devices 130 at the merchant system location.

[0066] At block 320, a merchant system operator installs the payment application 133 on the merchant point-of-sale device 130. In another exemplary embodiment, the merchant system operator purchases a merchant POS device 130 from the payment processing system 160 with the payment application 133 pre-installed on the merchant POS device 130. In an exemplary embodiment, the merchant POS device 130 may communicate with the payment processing system 160 via a network 120. In an exemplary embodiment, the merchant POS device 130 communicates with the payment processing system 160 via the payment application 133. For example, the merchant POS device 130 may be capable of sending transaction details to the payment processing system 160 via the payment application 133 over the network 120 to enable the payment processing system 160 to process a transaction.In another example, the merchant POS device 130 may be capable of receiving an acknowledgment from the payment processing system 160 that notifies a merchant POS device operator 102 whether or not a transaction was successful.

[0067] At block 330, the merchant beacon device 120 receives a beacon identifier from the payment processing system 160. In an exemplary embodiment, the merchant system receives a beacon identifier from the payment processing system 160 and installs or otherwise stores the beacon identifier on the merchant beacon device 120. In an exemplary embodiment, a merchant system operator installs the merchant beacon device 120 near a merchant POS device 130. In an exemplary embodiment, the merchant system operator installs multiple merchant beacon devices 120 and each merchant beacon device 120 near one or more connected merchant POS devices 130.In one exemplary embodiment, the merchant beacon device 120 may transmit a merchant beacon identifier over a wireless medium, wherein one or more user computing devices 110 located within a threshold proximity to the merchant beacon device 120 may receive the merchant beacon identifier over the wireless medium. In another embodiment, the merchant beacon device 120 may establish a local network 120 connection with one or more user computing devices 110 located within a threshold proximity to the merchant beacon device 120, and the merchant beacon device 120 sends the merchant beacon identifier to the one or more user computing devices 110 over the established local network 120 connection. The threshold proximity depends, for example, on the network 120 communication protocol used by the merchant beacon device 120.

[0068] At block 340, the merchant beacon device 120 transmits the beacon identifier via wireless communication at the merchant system location. The merchant beacon device 120 may, for example, transmit, emit, or otherwise send data comprising the beacon identifier to one or more user computing devices 110 located at the merchant system location within a threshold proximity to the merchant beacon device 120 via Wi-Fi, Bluetooth, Bluetooth Low Energy ("BLE"), near-field communication ("NFC"), or another suitable communication protocol. In some exemplary embodiments, the merchant beacon device 120 may be used at a time prior to transmitting the merchant beacon identifier to establish a network 120 connection between the merchant beacon device 120 and one or more user computing devices 110 located within a threshold proximity to the merchant beacon device 120.

[0069] At block 350, a merchant system operator installs the merchant camera device 140 at the merchant system location to correspond to the merchant beacon device 120. In one exemplary embodiment, both a merchant camera device 140 and a merchant beacon device 120 are installed proximate a particular merchant POS device 130. In another exemplary embodiment, a merchant camera device 140 and a merchant beacon device 120 are installed proximate two or more particular merchant POS devices 130. In one exemplary embodiment, a merchant camera device 140 is oriented to be capable of capturing video and / or images of a face of a user 101 standing in front of one or more merchant POS devices 130 during the checkout process.In one exemplary embodiment, the merchant system installs a merchant camera device 140 oriented to capture video and / or images of the face of a user standing in front of a particular merchant POS device 130. In another exemplary embodiment, the merchant system installs a merchant camera device 140 oriented to capture video and / or images of the faces of one or more users 101 standing within a proximity of a particular plurality of merchant POS devices 130 within a range of the field of view of the camera module 147 of the merchant camera device 140.

[0070] At block 360, the payment processing system 160 receives a merchant camera device 140 identifier and associates it with the corresponding beacon identifier of the merchant beacon device 120. In an exemplary embodiment, the merchant system and / or the payment processing system 160 configures the merchant camera device 140 so that the merchant camera device 140 can communicate with the payment processing system 160 over the network 120. An exemplary merchant camera device 140 identifier includes a hardware identifier, a Media Access Control (MAC) address, or other useful or relevant identifier associated with the merchant camera device 140. In an exemplary embodiment, the payment processing system 160 includes a database that includes the merchant camera device 140 identifiers and associated beacon identifiers for merchant beacon device 120 identifiers for a particular merchant system location.In an exemplary embodiment, the merchant camera device sends the merchant beacon device 120 identifier alongside the merchant camera device 140 identifier to the payment processing system 160. In an exemplary embodiment, during the setup and installation process, the merchant camera device 140 may receive the merchant beacon device 120 identifier from the merchant beacon device 120 via a suitable wireless communication channel. In an exemplary embodiment, during the setup and installation process, the merchant camera device 140 establishes a network 120 connection with the merchant beacon device 120 and receives the merchant beacon device 120 identifier via the network 120.In another exemplary embodiment, the payment processing system 160 receives the merchant camera device 140 identifier, extracts one or more merchant beacon device 120 identifiers from the database, and associates the merchant camera device 140 identifier with one or more of the extracted one or more merchant beacon device 120 identifiers. In yet another exemplary embodiment, the merchant system operator installs the one or more merchant beacon devices 120 after installing the one or more merchant camera devices 140. In this exemplary embodiment, the payment processing system 160 generates a merchant beacon device identifier to associate with a merchant camera device 140 identifier and sends the generated merchant beacon device identifier to the merchant system.In this exemplary embodiment, the merchant system operator configures the merchant beacon device 120 to transmit, emit, or otherwise send the merchant beacon device identifier assigned by the payment processing system 160 over a network 120.

[0071] In certain exemplary embodiments, the merchant camera device 140 and / or the merchant beacon device 120 are components of the merchant point-of-sale device 130 or are wirelessly or physically connected to the merchant POS device 130 and controlled by one or more processors of the merchant POS device 130. In certain exemplary embodiments, certain functions described herein as being performed by the merchant camera device 140 and / or the merchant beacon device 120 may also be performed by the merchant POS device 130.

[0072] From block 360, the method 210 proceeds to block 220 of Fig. 2 continued.

[0073] To Fig. 2 in block 220, the user 101 logs on to the payment processing system 160. The method for a user 101 to log on to an account with a payment processing system 160 is described below with reference to the Fig. 4 described methods 220 are described in more detail.

[0074] Fig. 4 is a block diagram illustrating a method 220 for a user 101 to register for an account with a payment processing system 160, according to certain exemplary embodiments. The method 220 is described with reference to the Fig. 1 illustrated components.

[0075] At block 410, user 101 accesses the payment processing system's web page 169. For example, user 101 accesses the payment processing system 160 via the web browser 118 of user computing device 110. In another example, user 101 may otherwise contact the payment processing system 160 to sign up for a user 101 account.

[0076] At block 420, user 101 logs on to payment processing system 160. User 101 may obtain a user account number, receive the appropriate applications and software to install on user computing device 110, request authorization to participate in hands-free payment processing, or perform any action required by payment processing system 160. User 101 may use the features of user computing device 110, such as user interface 115 and web browser 118, to log on to and configure a user 101 account.In an exemplary embodiment, user 101 may enter payment account information associated with one or more user 101 payment accounts, such as one or more credit accounts, one or more bank accounts, one or more stored value accounts, and / or other suitable accounts, into the user 101 account maintained by payment processing system 160.

[0077] At block 430, user 101 downloads payment application 113 to user computing device 110. In an exemplary embodiment, payment application 113 operating on user computing device 110 may be capable of communicating with payment processing system 160 via network 120. In an exemplary embodiment, user 101 may configure user 101 account settings or add, delete, or edit payment account information via payment application 113. In an exemplary embodiment, user 101 may select an option to enable or disable authorization of payment processing system 160 to process hands-free transactions.For example, a "hands free" transaction includes a transaction in which the user 101 is not required to interact with the user computing device 110 or that requires minimal user 101 interaction with the user computing device 110 to initiate a transaction with the merchant system.

[0078] At block 440, the payment processing system 160 sets up a face template associated with the user 101 account. The method for setting up a face template associated with a user 101 account is described below with reference to the Fig. 5 described methods 440 are described in more detail.

[0079] Fig. 5 is a block diagram illustrating a method 440 for setting up a face template associated with a user 101 account, according to certain exemplary embodiments. The method 440 is described with reference to the Fig. 1 illustrated components.

[0080] At block 510, the payment application 113 displays a request to the user 101 to capture a facial image via the user computing device 110. In an exemplary embodiment, the payment application 113 displays the request via the user interface 115. In one example, the user interface 115 may display a request such as: "To enable 'Hands Free' transactions, we need a picture of your face. Would you like to send a facial image now?" In this example, the user 101 may select an option to capture the current image, or may otherwise select an image stored on the user computing device 110.

[0081] At block 520, user 101 selects an option to capture a facial image. For example, user 101 actuates an object on user interface 115 that says, "Yes, I want to capture a picture now."

[0082] At block 530, the payment application 113 activates a camera module 117 on the user computing device 110, and the user 101 captures a facial image of themselves. In an exemplary embodiment, the user interface of the user computing device 115 may display a live camera feed of the user 101 to assist the user 101 in aligning the face of the user 101 to capture the facial image. In an exemplary embodiment, the payment application 113 on the user computing device 110 may display a box or outline on the user interface 115 within which the user 101 should align their face to capture an image of a required size predetermined by the payment processing system 160. In an exemplary embodiment, the user 101 may manipulate an object on the user interface 115 to capture the image.In this exemplary embodiment, in response to the user manipulating the object on the user interface 115, the camera module 117 receives a command from the payment application 113 to capture an image of the user 101. In another exemplary embodiment, the camera module 117 receives a command from the payment application 113 to capture a plurality of images of the user 101 as the user 101 moves the camera around the face of the user 101. Each of the plurality of images of the user 101 may, for example, correspond to a particular pose of the face of the user 101. An exemplary facial image may include a digital image of the face of a user 101. In an exemplary embodiment, the payment processing system 160 may establish guidelines for the user 101 when sending the facial images.For example, the payment application 113 may instruct the user 101 to remove any hats, headgear, glasses, or other objects or accessories that cover regions of the user's 101 face so that the payment application 160 can receive a full representation of the user's 101 face.

[0083] In an exemplary embodiment, the user computing device 110 determines whether the captured facial image is a valid facial image or an invalid facial image. For example, a valid facial image conforms to the guidelines previously determined by the payment processing system 160, and an invalid facial image does not conform to one or more of the guidelines. For example, if the user computing device 110 captures a facial image that includes incorrect dimensions, if part or all of the face of the user 101 is obscured, or if the image is too dark or too light, the user computing device 110 rejects the invalid facial image and displays a request instructing the user 101 to capture a subsequent facial image.In this example, user 101 captures a subsequent facial image via user computing device 110, and user computing device 110 sends the subsequent facial image to payment processing system 160 via network 120.

[0084] At block 540, the payment processing system 160 receives the facial image. In another exemplary embodiment, the payment processing system 160 receives a plurality of facial images of the user 101. For example, the payment application 113 sends the one or more facial images of the user 101 to the payment processing system 160 via the network 120. In an exemplary embodiment, the payment processing system 160 associates the one or more received facial images with the user 101 account. For example, the payment processing system 160 may identify the user 101 account to be associated with the one or more received images because the user 101 is currently logged into the payment application 113 on the user computing device 110 at the time the one or more facial images are sent to the payment processing system 160.In certain exemplary embodiments, the payment processing system 160 may determine whether the received facial image is a valid facial image or an invalid facial image. For example, a valid facial image conforms to all guidelines previously determined by the payment processing system 160, and an invalid facial image does not conform to one or more of the guidelines. For example, if a user 101 submits a facial image that includes incorrect dimensions, if part or all of the user's 101 face is obscured, or if the image is too dark or too light, the payment processing system 160 rejects the invalid facial image and sends a request to the user computing device 110 instructing the user 101 to capture a subsequent facial image, which is sent to the payment processing system 160.In this example, user computing device 110 receives and displays the request, user 101 captures a subsequent facial image via user computing device 110, and user computing device 110 sends the subsequent facial image to payment processing system 160 via network 120.

[0085] At block 550, the payment processing system 160 creates a face template associated with the user account 101 based on the received face image. In another exemplary embodiment, the payment processing system 160 creates a corresponding face template for each of a plurality of received face images associated with the user 101 account. In one example, the face template is of a predetermined size, for example, a 128-byte face template. In one example, the payment processing system 160 creates a face template that generates a computer code representation of the digital face image. The face template may, for example, describe important features of the face image of the user 101, such as shape, color, line, value, space, form, texture, or other useful or relevant features of the image or particular regions of the image.In one example, the facial template is generated by processing the facial image via a convolutional neural network. In an exemplary embodiment, the payment processing system 160 stores the generated facial template associated with the user 101 in a data storage unit 166 connected to the payment processing system 160. The payment processing system 160 database may, for example, include a table or other means that correlates each user 101 account identifier with an associated facial template of the user 101.

[0086] In another exemplary embodiment, after user computing device 110 captures one or more facial images of user 101, user computing device 110 generates one or more facial templates corresponding to one or more of the one or more facial images of user 101. In this exemplary embodiment, user computing device 110 sends the generated one or more facial templates to payment processing system 160 via network 120.

[0087]

[0100] At block 560, the payment processing system 160 deletes the received facial image. For example, the payment processing system 160 uses only a facial template comprising a computer code representation of the facial image of the user 101. In another example, the payment processing system 160 stores the received facial image for future processing. For example, at a later time, the payment processing system 160 updates a facial template generation algorithm and generates an updated facial template corresponding to the stored facial image.

[0088] From block 560, the method 440 proceeds to block 450 in Fig. 4 continued.

[0089] To Block 450 in Fig. 4, the payment processing system 160 sets up an audio template associated with the user 101 account. The method for setting up an audio template associated with a user 101 account is described below with reference to the Fig. 6 described methods 450 are described in more detail.

[0090] Fig. 6 is a block diagram illustrating a method 450 for setting up an audio template associated with a user 101 account, according to certain exemplary embodiments. The method 450 is described with reference to the Fig. 1 illustrated components.

[0091] At block 610, the payment application 113 displays a request to the user 101 to record an audio recording of the user's 101 voice via the user computing device 110. In an exemplary embodiment, the payment application 113 displays the request via the user interface 115. In one example, the user interface 115 may display a request such as: "To enable hands-free transactions, we need a recording of your voice. Would you like to send an audio recording now?" In this example, the user 101 may select an option to send a live audio recording, or may otherwise select a pre-recorded audio recording of the user 101 stored on the user computing device 110.

[0092] At block 620, user 101 selects an option to record an audio recording. For example, user 101 actuates an object on user interface 115 that says, "Yes, I want to send an audio recording now."

[0093] At block 630, the payment application 113 activates an audio module (not shown) on the user computing device 110, and the user 101 records an audio of the voice of the user 101. In an exemplary embodiment, the user interface of the user computing device 115 may display guidelines received from the payment processing system 160 to assist the user 101 in sending an audio recording. For example, the payment application 113 may display instructions for the user 101 to record the voice of the user 101 speaking the name of the user 101. In another example, the payment application 113 instructs the user 101 to record an audio recording to establish a voice password that can be used in hands-free transactions by the user 101.In an exemplary embodiment, user 101 may manipulate an object on user interface 115 to capture or record a live audio recording. In this exemplary embodiment, in response to the user manipulating the object on user interface 115, the audio module receives a command from payment application 113 to capture an audio recording of user 101. In an exemplary embodiment, payment processing system 160 may establish guidelines for user 101 when sending the audio recordings. For example, payment application 113 may instruct user 101 to send an audio recording that is no longer than a predefined maximum duration. For example, the audio recording may not exceed five seconds.In another example, payment application 113 may instruct user 101 to send an audio recording lasting a predefined period of time, for example, an audio recording lasting five seconds. In an exemplary embodiment, after user 101 initiates an audio recording by actuating an object on user interface 115, payment application 113 sends a command to the audio module to stop recording user 101's voice after the predefined period of time or the predefined maximum period of time has elapsed.

[0094] At block 640, the payment processing system 160 receives an audio recording of the voice of the user 101. The payment application 113 or audio module, for example, sends the audio recording to the payment processing system 160 over the network 120. In another example, the payment application 113 retrieves an audio recording selected by the user 101 and stored in the data store 116 and sends the retrieved audio recording over the network 120 to the payment processing system 160.

[0095] At block 650, the payment processing system 160 creates an audio template associated with the voice of the user 101 based on the received voice of the user. In one example, the audio template has a predetermined size. In one example, the payment processing system 160 generates an audio template that includes a computer code representation of the audio recording of the user 101. The audio template may, for example, describe important features of the audio recording of the user 101, such as the intonation of the voice of the user 101 or other features of the voice of the user 101.

[0096] [Inventor: Does this work similarly to the facial template?] In an exemplary embodiment, the payment processing system 160 stores the generated audio template associated with the user 101 in a data storage unit 166 connected to the payment processing system 160. The payment processing system 160 database may include, for example, a table or other means that correlates each user 101 account identifier with an associated audio template of the user 101.

[0097] At block 660, the payment processing system 160 deletes the received audio of the voice of the user 101. In an exemplary embodiment, the payment processing system 160 deletes the received audio recording of the user 101 to protect the privacy of the user 101. For example, the payment processing system 160 uses only an audio template that includes a computer code representation of the audio recording of the user 101.

[0098] From block 660, the method 440 proceeds to block 230 in Fig. 2 continued.

[0099] To Block 230 in Fig. 2, user 101 enters the merchant system location and logs into payment application 113 on user computing device 110. In another exemplary embodiment, user 101 logs into payment application 113 at a time prior to entering the merchant system location and enters the merchant location carrying user computing device 110 logged into payment application 113.

[0100] At block 240, the user device receives a merchant beacon device 120 identifier. The method for receiving, by a user computing device 110, a merchant beacon identifier transmitted by a merchant beacon device 120 is described below with reference to the Fig. 7 described methods 240 are described in more detail.

[0101] Fig. 7 is a block diagram illustrating a method 240 for receiving a merchant beacon identifier transmitted by a merchant beacon device 120 by a user computing device 110, according to certain exemplary embodiments. The method 240 is described with reference to the Fig. 1 illustrated components.

[0102] At block 710, user 101 logs into payment application 113 on user computing device 110. In an exemplary embodiment, user 101 may have a username and password associated with user 101's account, which are maintained by payment processing system 160. In an exemplary embodiment, user 101 opens payment application 113 on user computing device 110 and enters a username and / or password via user interface 115 to log into payment application 113. In an exemplary embodiment, when user 101 is logged into payment application 113, the payment application may communicate with payment processing system 160 via network 120.In this exemplary embodiment, if user 101 is not logged in to payment application 113, the payment application does not communicate with payment processing system 160, even if the network 120 connection is available. In an exemplary embodiment, user 101 may log out of payment application 113 at any time by manipulating one or more objects on user interface 115 of user computing device 110. In an exemplary embodiment, after logging in to payment application 113, user 101 configures one or more user 101 account settings, adds, edits, or deletes user 101 payment account information, and / or changes user 101 preferences. In certain exemplary embodiments, user 101 may be required to make feature selections to obtain the benefits of the techniques described herein.For example, User 101 may need to enable one or more User 101 account settings to enable "Hands Free" transactions in accordance with the methods described herein.

[0103] In an exemplary embodiment, payment application 113 may provide user 101 with options, data, configurable messages, and other suitable features. For example, payment application 113 may include a listing of merchant systems and merchant locations that participate in hands-free payment transactions according to one or more of the methods described herein. The listing may be periodically updated by payment processing system 160. Payment application 113 may notify user 101 when user 101 is within a configured environment of a participating merchant system. Payment application 113 may provide user 101 with options for updating payment preferences. Payment application 113 may provide user 101 with a listing of recent transactions.The payment application 113 may provide the user 101 with any other suitable information.

[0104] At block 720, user 101 carries user computing device 110 within a threshold distance of a merchant beacon device 120 at the merchant system location. In one exemplary embodiment, user 101 enters a merchant system location. User 101 may enter the merchant location carrying user computing device 110 in a pocket or bag, in user 101's hands, or in any suitable manner. The merchant system location may be a store location, a kiosk location, or any suitable physical location of a merchant system. In another exemplary embodiment, merchant POS device operator 102 may be mobile and arrive at user 101's location. For example, the merchant system may be a restaurant, and merchant POS device operator 102 may be a delivery person possessing a portable merchant POS device 130.

[0105] In certain exemplary embodiments, payment application 113 may notify user 101 when user 101 is near a merchant system that accepts hands-free payments. The notification may be provided via a message on user computing device 110, in an email or text, or in any suitable manner. In one exemplary embodiment, the notification may be based on the location of user 101 as determined by a GPS module (not shown) located on user computing device 110. For example, payment application 113 accesses GPS data from the GPS module and compares the GPS location to a list of merchant system locations that accept hands-free payments. For example, payment application 113 includes or accesses a list maintained by payment processing system 160 of merchant system locations that accept hands-free payments.If a match results from the comparison, a notification is generated and provided to user 101. The match may result when user 101 is within a configured distance of a qualified merchant system location. In an exemplary embodiment, the notifications may be configured to notify in any suitable manner. In one example, the notifications may be combined in commercially densely populated environments, or the notifications may be presented individually. In another example, the notifications may be configured to notify user 101 at a configured frequency. For example, a notification may be presented three times, but on a fourth instance, the notification is not presented.The messages may be presented as a notification with an audible message, a vibration, a pop-up message on the user interface 115 of the user computing device 110, or any other suitable message.

[0106] At block 730, the user computing device 110 receives a beacon identifier transmitted from the merchant beacon device 120. The user computing device 110 detects a merchant beacon device 120 via wireless communication at the merchant system location. The user computing device 110 may be configured to scan for beacons or other wireless signals. In one exemplary embodiment, the user computing device 110 and the merchant beacon device 120 establish a wireless Wi-Fi network 120 connection. In another exemplary embodiment, the user computing device 110 and the merchant beacon device 120 establish a Bluetooth, BLE, NFC, or other suitable network 120 connection. After entering the signal range of the merchant beacon device 120, the user computing device 110 receives the merchant beacon identifier.

[0107] At block 740, user computing device 110 sends the received merchant beacon identifier and a user 101 account identifier to payment processing system 160. In an exemplary embodiment, user computing device 110 sends the data received in the merchant beacon identifier along with a user 101 account identifier to payment processing system 160 over network 120.

[0108] At block 750, the payment processing system 160 receives the merchant beacon identifier and the user 101 account identifier. For example, the payment processing system 160 receives the merchant beacon identifier and the user 101 account identifier via the network 120. The user computing device 110 may compare the data from the merchant beacon identifier with a database of merchant beacon identifier data and merchant camera device identifier data to determine an identity of the merchant system and merchant camera device 140 associated with the merchant beacon identifier and / or to verify the authenticity of the beacon.

[0109] From block 750, the method 240 proceeds to block 250 in Fig. 2 continued.

[0110] To Fig. 2, returning to block 250, the merchant camera device 140 receives a face template for each user 101 in range of the merchant beacon device 120. The method for receiving, by a merchant camera device 140, a face template for each user 101 in range of the merchant beacon device 120 is described below with reference to the method described in Fig. 8 described methods 250 are described in more detail.

[0111] Fig. 8 is a block diagram illustrating a method 250 for receiving a facial template for each user 101 in a range of the merchant beacon device 120 by a camera device 140, according to certain exemplary embodiments. The method 250 is described with reference to the Fig. 1 illustrated components.

[0112] At block 810, the payment processing system 160 extracts a facial template associated with the user 101 account identifier. In an exemplary embodiment, the payment processing system 160 accesses a database that includes stored facial templates of a plurality of users 101 with corresponding user 101 account identifiers for each user 101. This database is stored, for example, in the data storage unit 166.

[0113] At block 820, the payment processing system 160 identifies a merchant camera device 120 associated with the merchant beacon device 120 identifier. In an exemplary embodiment, the payment processing system 160 identifies that the merchant beacon identifier is associated with the payment processing system 160 and a particular merchant camera device 140 at the merchant system location. In an exemplary embodiment, the payment processing system 160 identifies that the merchant beacon identifier is associated with a plurality of merchant camera devices 140 installed at a particular merchant location.

[0114] At block 830, the payment processing system 160 sends the facial template of the identified user 101 to the merchant camera device 140 associated with the merchant beacon device identifier 120. In another exemplary embodiment, the payment processing system 160 sends the facial template of the identified user 101 to a plurality of merchant camera devices 140 associated with the merchant beacon device identifier 120. In certain exemplary embodiments, the payment processing system 160 receives, in real time, a plurality of transmissions from the user computing device 110 corresponding to a plurality of users 101 present at the merchant system location, each transmission including a user 101 account identifier and a rebroadcast merchant beacon identifier.In these exemplary embodiments, in response to each such transmission, the payment processing system 160 retrieves a facial template associated with the received user 101 account identifier and transmits a facial template to one or more merchant camera devices 140 at the merchant location associated with the merchant beacon identifier.

[0115] At block 840, the merchant camera device 140 receives the facial template of the user 101. In another exemplary embodiment, a plurality of merchant camera devices 140 receive the facial template of the user 101. In yet another exemplary embodiment, the merchant camera device 140 and / or the plurality of merchant camera devices 140 receive one or more additional facial templates from the payment processing system 160 corresponding to one or more users 101 other than the current user 101 having user computing devices 110 in network 120 communication with a merchant beacon device according to the methods described herein.The one or more additional facial templates are received by the payment processing system 160 in real time as additional users 101 other than the current user 101 receive the merchant beacon device 120 identifier over a wireless communications network 120 or otherwise establish a network 120 connection between their user computing devices 110 and one or more merchant beacon devices 120. For example, the one or more merchant camera devices 140 may receive one or more additional facial templates corresponding to one or more additional users 101 before, simultaneously with, or after the time the merchant camera device 140 receives the facial template of the current user 101.

[0116] At block 850, the merchant camera device 140 adds the facial template of the user 101 to a current customer log. In one exemplary embodiment, the current customer log is accessible by the merchant camera device 140 and by the payment processing system 160. In another exemplary embodiment, the current customer log is additionally accessible by the merchant POS device 130 and / or an applicable computing device of the merchant system. In one exemplary embodiment, the payment processing system 160 includes and maintains the current customer log, which is accessible by the merchant camera device 140 via the network 120. In another exemplary embodiment, the merchant camera device 140 includes the current customer log, which is accessible to the payment processing system 160 through communication with the user computing device 110 via the network 120.

[0117] At block 860, the merchant camera device 140 periodically updates the current customer log. The method for receiving, by a merchant camera device 140, a notification from a payment processing system 160 when users 101 enter or leave a network area of a merchant beacon device 120 is described below with reference to the Fig. 9 described methods 860 are described in more detail.

[0118] Fig. 9 is a block diagram illustrating a method 860 for receiving a notification from a payment processing system 160 through a merchant camera device 140 when users 101 enter or exit a network area of a merchant beacon device 120, according to certain exemplary embodiments. The method 860 is described with reference to the Fig. 1 illustrated components.

[0119] At block 910, the merchant camera device 140 is notified by the payment processing system 160 when users 101 enrolled in a payment account enter or exit a network range of the merchant beacon device 120. For example, as previously discussed, when a user 101 carrying a user computing device 110 enters a threshold distance from a merchant beacon device 120, the merchant beacon device 120 or the user computing device 110 of the user 101 may detect the other device and establish a wireless network 120 connection between the two devices at the merchant system location. In this example, the merchant beacon device 120 sends the merchant beacon identifier corresponding to the merchant beacon device 120 to the user computing device 110 over the wireless network 120. For example, the merchant beacon device 120 sends the merchant beacon identifier to the user computing device 110 over a wireless communication network 120 including Wi-Fi, Bluetooth, BLE, or NFC.In this example, user computing device 110 retransmits the received merchant beacon identifier to payment processing system 160 along with a user 101 account identifier that identifies user 101.

[0120] At block 920, the merchant camera device 140 determines whether a new user 101 is within range of the merchant beacon device 120. For example, if the payment processing system 160 receives a new user 101 account identifier along with the same merchant beacon identifier, the payment processing system 160 may determine that a new user 101 is within range of the merchant beacon device 120. In this example, the payment processing system 160 may infer that the new user 101 has entered the merchant location based on receiving the new user 101 account identifier. In another example, if the payment processing system 160 does not receive any new user 101 account identifiers along with the same merchant beacon identifier within a threshold time period, the payment processing system 160 may determine that no new users 101 have entered the network 120 range of the merchant beacon device 120.

[0121] If a new user 101 is within range of the merchant beacon device 120, the method 860 proceeds to block 830 in Fig. 8. For example, the payment processing system 160 receives a new user 101 account identifier alongside the same merchant beacon identifier. In this example, the payment processing system 160 infers that the new user 101 has entered the merchant location based on the receipt of the new user 101 account identifier and the same merchant beacon identifier as previously received from the first user 101.

[0122] To Fig. 8, in block 810, the payment processing system 160 extracts a facial template associated with the new user 101 account identifier. In an exemplary embodiment, the payment processing system 160 sends the facial template to the corresponding one or more merchant camera devices 140, and the one or more merchant camera devices 140 add the user 101 facial template to the current customer log according to the exemplary method previously described in method 250 in Fig. 8 was described.

[0123] To Fig. 9 Returning to block 920, if no new user 101 is within range of the merchant beacon device 120, the method 860 proceeds to block 930. For example, the payment processing system 160 receives no new user 101 account identifiers along with the same merchant beacon identifier within a threshold time period and determines that no new users 101 have entered the network 120 range of the merchant beacon device 120.

[0124] At block 930, the merchant camera device 140 determines whether a current user 101 has moved out of range of the merchant beacon device 120. In an exemplary embodiment, the user computing device 110 continues to receive the merchant beacon identifier from the merchant beacon device 120 and retransmits the merchant beacon identifier along with the user 101 account identifier to the payment processing system 160. In this exemplary embodiment, the user computing device 110 may periodically transmit information including the merchant beacon identifier and the user 101 account identifier to the payment processing system 160 as long as the user computing device 110 continues to detect the merchant beacon device 120 and receive the merchant beacon device 120 identifier via periodic scans. For example, the user computing device scans for the merchant beacon device 120 every five seconds.In another exemplary embodiment, the user computing device 110 may periodically send information including the merchant beacon identifier and the user 101 account identifier to the payment processing system 160 as long as the user computing device 110 maintains a wireless network 120 connection with the merchant beacon device 120. For example, the user computing device 110 may send this information to the payment processing system 160 every five seconds. In this exemplary embodiment, if the payment processing system 160 stops receiving information from the user computing device for a predefined number of intervals, the payment processing system 160 may determine that the corresponding user 101 has moved out of range of the merchant beacon device.In this exemplary embodiment, if the payment processing system 160 continues to receive information sent from the user computing device 110 at expected intervals, the payment processing system 160 determines that the user 101 is still within the network 120 range of the merchant beacon device 120.

[0125] If no current user 101 has moved out of range of the merchant beacon device 120, the method 860 continues with block 260 in Fig. 2. For example, the payment processing system 160 continues to receive the merchant beacon identifier and the user 101 account identifier sent from the user computing device 110 at expected intervals and determines that the user 101 is still within the network 120 range of the merchant beacon device 120.

[0126] To Fig. 2 Returning to block 260, user 101 initiates a transaction at merchant POS device 130.

[0127] To Fig. 9 Returning to block 930, if a current user 101 has moved out of range of the merchant beacon device 120, the method 860 continues to block 940.

[0128] At block 940, the merchant camera device 140 receives a notification from the payment processing system 160 that a current user 101 is outside the network range of the merchant beacon device 120. In another exemplary embodiment, the merchant camera device 140 receives a notification from the payment processing system 160 that the user computing device 110 associated with the current user 101 has stopped sending notifications to the payment processing system 160 that include the merchant beacon device 120 identifier.For example, the merchant camera device 140 receives the user 101 account identifier associated with the current user 101 connected to a user computing device 110 that is either out of network range or has stopped sending notifications including the merchant beacon device 120 identifier to the payment processing system 160, accesses the current customer log, and finds an entry corresponding to the current user 101. The current customer log is maintained, for example, by the payment processing system 160, and the merchant camera device 140 accesses the current customer log over the network 120 by communicating with the payment processing system 160. In another exemplary embodiment, the payment processing system 160 does not send a notification to the merchant camera device 140 that the current user 101 is out of network range.In this exemplary embodiment, the payment processing system 160 accesses the current customer log and deletes the face template of the current user 101 from the current customer log.

[0129] At block 950, the merchant camera device 140 deletes the face template of the current user 101 from the current customer log. The current customer log includes, for example, a table, and the merchant camera device 140 deletes or requests deletion of an entry or row corresponding to data associated with the current user 101 for whom the merchant camera device 140 received the notification. In another exemplary embodiment, the payment processing system 160 accesses the current customer log and deletes the face template of the current user 101 from the current customer log.

[0130] From block 950, the method 860 proceeds to block 260 in Fig. 2 continued.

[0131] To Fig. 2 in block 260, the user 101 initiates a transaction at the merchant point of sale device 130. The method for initiating a transaction at a merchant point of sale device 130 by a user 101 is described below with reference to the method described in Fig. 10. In the exemplary embodiment described herein, user 101 initiates a hands-free transaction at merchant POS device 130. An exemplary hands-free transaction requires no interaction with computing device 110 by user 101. In another example, the hands-free transaction requires only minimal interaction with user computing device 110 by user 101.

[0132] Fig. 10 is a block diagram illustrating a method 260 for initiating a transaction at a merchant point-of-sale device 130 by a user 101 according to certain exemplary embodiments. The method 260 is described with reference to the Fig. 1 illustrated components.

[0133] At block 1010, user 101 approaches a merchant point-of-sale device 130. In an exemplary embodiment, at a time prior to approaching the merchant POS device 130, user 101 browses the merchant system location and selects one or more items for purchase. In this exemplary embodiment, user 101 may collect the one or more items and carry or otherwise transport them to the merchant POS device 130 using a physical basket or shopping cart.

[0134] At block 1020, the merchant point-of-sale device 130 operator 102 determines the total amount of the purchase items of the user 101. In an exemplary embodiment, the merchant POS device operator 102 scans barcodes affixed to the one or more items or otherwise enters descriptions and prices associated with the one or more items into the merchant POS device 130. In an exemplary embodiment, after scanning or manually entering the items into the merchant POS device 130, the merchant POS device operator 102 actuates an object on the user interface 135 of the merchant POS device 130 to instruct the merchant POS device 130 to determine the total amount for the items. In an exemplary embodiment, the merchant POS device 130 displays the total amount to the user 101 via the user interface 135.

[0135] At block 1030, the merchant point-of-sale device 130 operator asks the user 101 to select a payment option. In an exemplary embodiment, the merchant POS device 130 displays one or more payment options that the user 101 can select for use in a transaction. Example payment options may include payment via a payment application 113 connected to the payment processing system 160, payment with cash, payment with a check, card payment, payment with a stored-value card, and / or any other payment method that the merchant system is capable of accepting or willing to accept for payment by the user 101.In an exemplary embodiment, the one or more payment options are displayed as objects on the user interface 135 and are selectable by the merchant POS device operator 102 in response to the user 101 instructing the merchant point of sale device 102 operator to make a selection.

[0136] In another exemplary embodiment, the payment processing system 160 or the merchant camera device 140 sends a notification to the merchant point-of-sale device 140 that the user 101 may initiate a transaction via the payment application 113. In this exemplary embodiment, the merchant point-of-sale device 140 notifies the merchant point-of-sale device 140 operator that the user 101 may participate in a transaction using the payment application 113. In this exemplary embodiment, the merchant point-of-sale device 140 operator asks the user 101 to confirm whether the user 101 wishes to initiate a transaction using the payment application 113.

[0137] At block 1040, user 101 instructs merchant POS device operator 102 to initiate a transaction with payment application 113. In an exemplary embodiment, in response to receiving a verbal request from user 101 to select payment application 113 as the payment option, merchant POS device operator 102 actuates an object on user interface 135 of merchant POS device 130 corresponding to the payment application 113 payment option.

[0138] At block 1050, the merchant point-of-sale device operator 102 selects an option on the merchant point-of-sale device 130 to initiate a transaction using the payment application 113. In an exemplary embodiment, the merchant POS device 130 displays a confirmation screen after the merchant POS device operator 102 selects an option to initiate a transaction using the payment application 113. An exemplary confirmation screen may display information summarizing the potential transaction and may include one or more of a transaction total, a description of the one or more items purchased by the user 101, and an indication that the user 101 selected the payment application 113 as the payment method for the transaction.An exemplary confirmation screen may display further options to confirm the transaction or cancel the transaction. In an exemplary embodiment, user 101 reviews the confirmation screen, determines that the information displayed on the confirmation screen is correct, determines to proceed with the transaction, and instructs merchant POS device operator 102 to select the option to confirm the transaction via user interface 135.

[0139] In another exemplary embodiment, user 101 decides to discard the transaction because the information is incorrect, or because user 101 has changed their mind and decided not to purchase the items. In yet another exemplary embodiment, the confirmation screen further includes an option to edit the transaction details. In this exemplary embodiment, merchant POS device operator 102, upon instruction from user 101, may select the option to edit the transaction details and may then edit, add, or delete one or more of the items in the transaction, or edit payment details or payment methods for the transaction.

[0140] At block 1060, the merchant point of sale device 130 sends a request to identify the user 101 to the payment processing system 160. In an exemplary embodiment, the merchant point of sale device sends a request to process a transaction along with the request to identify the user 101.

[0141] At this point, for example, the payment processing system 160 knows that one or more users 101 are physically present at the merchant system location based on the current customer log maintained by the payment processing system and accessible and editable by the merchant camera device 120. Furthermore, the payment processing system 160 knows that a user 101 initiated a transaction using the payment method of the payment application 113 based on the request to process the transaction received from the merchant POS device 130. However, the payment processing system 160 may not know exactly which user 101 requested processing of the hands-free transaction involving the payment application 113.

[0142] At block 1070, the payment processing system 160 receives the request to identify the user 101 from the merchant point of sale device 130. For example, the payment processing system 160 receives the request to identify the user and the request to process the transaction over the network 120.

[0143] At block 1080, the payment processing system 160 sends a request to the merchant camera device 140 to identify the user 101. For example, the payment processing system 160 sends the request to identify the user to the merchant camera device 140 via the network 120. In another exemplary embodiment, the merchant camera device 140 has already communicated the identity of the user 101 to the payment processing system 160 or the merchant point of sale device 130, and the payment processing system 160 need not send a request to the merchant camera device 140 to identify the user 101.

[0144] From block 1080, the method 260 proceeds to block 270 in Fig. 2 continued.

[0145] To Fig. 2, in block 270, the merchant camera device 140 identifies the user 101 via facial recognition. The method for identifying a user 101 by a merchant camera device 140 via facial recognition is described below with reference to the Fig. 11 described methods 270 are described in more detail.

[0146] Fig. 11 is a block diagram illustrating a method 270 for identifying a user 101 via facial recognition by a merchant camera device 140, according to certain exemplary embodiments. The method 270 is described with reference to the Fig. 1 illustrated components.

[0147] At block 1110, the merchant camera device 140 captures video of the user 101. In one exemplary embodiment, in response to receiving a request to identify the user 101, the merchant camera device 140 activates the camera module 147 to begin capturing a video of the surroundings of the merchant camera device 140. In one exemplary embodiment, the merchant POS device 130 captures a video feed of the face of the user 101. In another exemplary embodiment, the merchant camera device 140 continuously captures, but does not record, a video feed of its surroundings. In this exemplary embodiment, when the merchant camera device 140 receives a request to identify the user 101 from the payment processing system 160, the merchant camera device begins recording the video feed for a threshold period of time.In an exemplary embodiment, the user 101 may be moving during the period in which the merchant camera device 140 records the video feed. In an exemplary embodiment, a processor 143 of the merchant camera device 140 extracts a facial image by determining a particular frame of the video feed and the region of the video feed instance that corresponds to the user's face.

[0148] At block 1120, the merchant camera device 140 extracts a facial image of the user 101 from the captured video. In an exemplary embodiment, the merchant camera device 140 determines a frame of the captured video to provide an image of the face of the user 101 and extracts the frame of the captured video that includes the facial image of the user 101.

[0149] In certain other embodiments, the merchant camera device 140 determines a frame of the captured video to provide an image of the faces of the plurality of users 101. For example, the frame includes an image of the face of a first user 101, a second user 101, and a third user 101 at different locations in the image. In this example, a merchant camera device 140 may capture video of an environment corresponding to an area near multiple merchant POS devices 130. In this exemplary embodiment, the merchant camera device 140 may determine to which particular merchant POS device 130 each of the plurality of faces of the corresponding plurality of users 101 in the extracted image corresponds.

[0150] In an exemplary embodiment where a merchant camera device 140 is mounted at an angle parallel to a plurality of adjacent merchant POS devices 130, the field of view of the merchant camera device 140 may be divided into bounding boxes such that each bounding box is assigned to a specific merchant POS device 130. In this exemplary embodiment, the bounding boxes may overlap because one or more users 101 may be located between any two adjacent merchant POS devices 130. In this exemplary embodiment, each bounding box is assigned to at most two adjacent merchant point-of-sale devices 130.In this exemplary embodiment, where the merchant camera device 140 is mounted parallel to a plurality of adjacent merchant POS devices 130, the size of a detected facial image of a user 101 is directly proportional to the distance the user 101 is from a merchant POS device 130 detected in the image. In this exemplary embodiment, the merchant camera device 140 assigns one or more specific merchant POS devices 130 to each detected facial image based on the position of the facial image relative to the bounding boxes.

[0151] In another exemplary embodiment, the merchant camera device 140 is mounted at an angle that is not parallel to a plurality of adjacent merchant POS devices 130. In this exemplary embodiment, the merchant camera device 140 may be capable of monitoring an environment that includes a larger number of adjacent merchant POS devices 130 than the exemplary embodiment in which the merchant camera device 140 is mounted parallel to the adjacent merchant POS devices 130. However, in this exemplary embodiment, the size of a detected facial image is not proportional to the distance between a merchant POS terminal detected in the image and the detected facial image. In this exemplary embodiment, the merchant camera device 140 assigns one or more specific merchant POS devices 130 to each detected facial image based on the position of the facial image relative to the bounding boxes.

[0152] At block 1130, the merchant camera device 140 generates a face template from the captured facial image. In one example, the face template is of a predetermined size, for example, a 128-byte face template. In one example, the payment processing system 160 generates a face template that generates a computer code representation of the digital facial image. The face template may, for example, describe important features of the facial image of the user 101, such as shape, color, line, value, location, form, texture, or other useful or relevant features of the image or specific regions of the image. In another example, the face template is generated by processing the facial image via a convolutional neural network. In one exemplary embodiment, the merchant camera device 140 stores the generated face template in a data storage unit 146 coupled to the merchant camera device 140.For example, the merchant camera device 140 database may include a log of current customer facial templates in which the merchant camera device 140 stores the generated facial templates.

[0153] In certain other exemplary embodiments, the merchant camera device 140 continuously records a video feed of their surroundings as users 101 enter or exit the vicinity of one or more merchant POS devices 130 over a given period of time. In this exemplary embodiment, the processor 147 of the merchant camera device 140 may continuously monitor the incoming video feed to detect faces from extracted frames of the video feed. In this exemplary embodiment, each time the processor 147 detects the presence of one or more faces in the video feed, the merchant camera device 140 extracts a frame of the video feed that includes one or more facial images of one or more detected faces and creates facial templates based on the extracted one or more facial images.In this exemplary embodiment, the merchant camera device 140 stores face templates in the current customer face template history as they are generated. In this exemplary embodiment, as the merchant camera device 140 generates subsequent face templates, the merchant camera device 140 determines whether the generated subsequent face template is similar within a threshold compared to any of the face templates already stored in the current customer face template history.If the generated subsequent face template is similar within a threshold to any of the face templates already stored in the log, the merchant camera device 140, after connecting the face template to one or two specific merchant POS devices 130 based on the position of the connected face templates in the extracted frame of the captured video, adds the face template to the current customer's face template log. If the generated subsequent face template is not similar within a threshold to any face templates already stored in the current customer's face template log, the merchant camera device 140 deletes the generated face template or otherwise ignores it and / or does nothing.In this exemplary embodiment, when the merchant camera device 140 determines that the particular facial image is no longer in the field of the video feed, the corresponding facial template is deleted from the current customer's facial template history.

[0154] At block 1140, the merchant camera device 140 deletes the captured video and the extracted facial image. For example, the merchant camera device 140 does not store captured images or videos. In this example, facial templates generated by the merchant camera device 140 include computer code representations of the facial images of the users 101. In this example, after generating a facial template or after a threshold time has elapsed after capturing videos or images or extracting an image from a video, the merchant camera device 140 deletes all captured or extracted videos or images.

[0155] At block 1150, the merchant camera device 140 retrieves facial templates from the current customer log. For example, the current customer log includes facial templates received from the payment processing system 160 corresponding to all current users 101 whose connected user computing devices 110 are within a network distance of a merchant beacon device 120.

[0156] At block 1160, the merchant camera device 140 compares the generated facial template from the captured facial image with facial templates from the current customer log.

[0157] At block 1170, the merchant camera device 140 determines whether a match exists between the generated facial template and one of the facial templates from the current customer log. In this exemplary embodiment, if the merchant camera device 140 determines a match between a first facial template in the current customer facial template log and a second facial template of the current customer log, the merchant camera device 140 determines which one or two merchant point-of-sale devices 130 the second facial template is associated with. In this exemplary embodiment, the merchant camera device 140 updates the second facial template entry in the current customer log with an annotation associating the second facial template with specific one or two merchant POS devices 130.For example, as previously discussed, the facial image over which the facial template was located is within a bounding box of a portion of the video or image corresponding to one or two particular merchant POS devices 130. In this example, when the merchant camera device 140 generates a facial template for the facial image, the merchant camera device 140 associates the generated facial template with the particular one or more merchant POS devices 130. For example, if the facial image was in a bounding box in front of a first POS device 130, the resulting facial template may be associated with the first merchant POS device 130. In another example, if the facial image was in a bounding box between the first merchant POS device 130 and a second, adjacent POS device 130, the resulting facial template may be associated with both the first merchant POS device 130 and the second merchant POS device 130.

[0158] If a face template from the current customer log matches the generated face template, the method 270 proceeds to block 1180. In another exemplary embodiment, if a face template from the current customer log matches a face template from the current customer face template log, the method 270 proceeds to block 1180.

[0159] At block 1180, the merchant camera device 140 sends the facial template from the current customer log corresponding to the identified user to the payment processing system 160. The sent facial template includes, for example, an annotation that assigns the facial template to specific one or two merchant POS devices 130.

[0160] From block 1180, the method 270 proceeds to block 295 in Fig. 2. In this exemplary embodiment, the payment processing system 160 identifies the user 101 requesting a transaction and a particular merchant POS device 130 with which the user 101 is associated and may proceed with processing the transaction.

[0161] Returning to block 1170, if no face template from the current customer protocol matches the generated face template, the method 270 proceeds to block 280 in Fig. 2. In another exemplary embodiment, if no face template from the current customer log matches any face template from the current customer face template log, the method 270 continues with block 280 in Fig. 2 continued.

[0162] To Block 280 in Fig. 2, the payment processing system 160 identifies the user 101 via speech recognition. In another exemplary embodiment, the payment processing system 160 does not identify the user 101 via speech recognition and continues with the identification of the user 101 via a question and answer. For example, the merchant camera device 140 and / or the payment processing system 160 may not identify the user via speech recognition and / or if the merchant camera device 140 and / or the payment processing system 160 identifies two or more users 101 via facial recognition. The method for identifying a user 101 by a payment processing system 160 via speech recognition is described below with reference to the Fig. 12 described procedures 280 are described in more detail.

[0163] Fig. 12 is a block diagram illustrating a method 280 for identifying a user 101 via speech recognition by a payment processing system 160 according to certain exemplary embodiments. The method 280 is described with reference to the Fig. 1 illustrated components.

[0164] In certain exemplary embodiments, it may be necessary to identify a user 101 via voice recognition because the merchant camera device 140 and / or the payment processing system 160 cannot identify the user 101 based on facial recognition. The merchant camera device 140 may not be able to extract an appropriate facial image of the user 101 from the video feed to generate a facial template. In another example, the merchant camera device 140 cannot find a match for the facial template of the user 101 in the current customer log. In yet another example, the merchant camera device 140 identifies a matching user 101 facial template in the current customer log, but the identified user 101 facial template is assigned to two neighboring merchant POS devices 130.In this example, user 101 may need to identify themselves via voice recognition at the point of sale device 130 at which user 101 initiates the transaction.

[0165] At block 1210, the merchant camera device 140 sends an indication that the user 101 cannot be identified to the payment processing system 160. In another exemplary embodiment, the merchant camera device 140 sends an indication that the user 101 has been identified, but the merchant camera device 140 cannot determine which of the two adjacent merchant POS devices 130 the user 101 is using in the transaction.

[0166] At block 1220, the payment processing system 160 retrieves audio templates corresponding to users 101 from the current customer log. As previously discussed, when a user 101 sets up an account with the payment processing system 160, the user 101 may send an audio recording of the user 101's voice to the payment processing system 160. In this example, the payment processing system 160 sets up an audio template corresponding to the user 101 based on the received audio recording. In other exemplary embodiments, the user 101 does not send an audio recording to the payment processing system 160 at the time the user 101 sets up the user 101 account. In this exemplary embodiment, the payment processing system 160 does not have an audio template associated with the user 101 account and cannot verify the user 101 via speech recognition.

[0167] At block 1230, the payment processing system 160 sends a request to the merchant system point-of-sale device 130 to record audio from the user 101. The payment processing system 160 sends the request, for example, over the network 120.

[0168] At block 1240, the merchant system point-of-sale device 130 displays the request for user 101 to record audio. For example, the merchant system point-of-sale device 130 may display instructions for user 101 to record an audio recording. For example, the user 101 may be instructed to speak the same words in the same intonation as the user 101 did when setting up the audio template with the payment processing system 160 at the time of setting up the user 101 account.

[0169] At block 1250, the merchant system point-of-sale device 130 records a voice input from the user 101 and sends the voice input to the payment processing system 160. For example, the merchant POS device operator 102 may actuate an object on the user interface 135 to activate an audio module 131 to receive an audio input from the user 101, which the merchant POS device 130 retransmits to the payment processing system 160 via the network 120. In an exemplary embodiment, the merchant POS device 130 transmits a merchant POS device 130 identifier along with the retransmitted user 101 audio input.

[0170] At block 1260, the payment processing system 160 compares the received voice input with the retrieved audio templates from the current customer log corresponding to users 101. For example, the payment processing system 160 receives the audio input of user 101 from the merchant POS device 130 over the network 120.

[0171] At block 1270, the payment processing system 160 determines whether a match exists between the received voice input and one of the retrieved audio templates from the current customer log. In an exemplary embodiment, the payment processing system 160 compares one or more features between the received voice input and each of the retrieved audio templates from the current customer log. In an exemplary embodiment, if the similarity between the received voice input and a particular audio template exceeds a predefined threshold, the payment processing system 160 determines that the received voice input matches the particular audio template.In another exemplary embodiment, if the similarity between the received speech input and the particular audio template is less than a predefined threshold, the payment processing system 160 determines that the received speech input does not match the particular audio template.

[0172] If an audio template from the current customer log matches the received voice input, method 270 proceeds to block 1280. For example, the similarity between a particular audio template from the current customer log and the received audio recording of user 101 exceeds a predefined threshold, and payment processing system 160 determines that a match exists.

[0173] At block 1280, the payment processing system 160 assigns the user 101 to the merchant point-of-sale device 130 in the current customer log. For example, the payment processing system 160 identifies the user 101 associated with the audio template that matches the received user 101 audio recording. Additionally, the payment processing system 160 identifies the merchant POS device 130 to which the user 101 should be assigned based on the merchant POS device 130 identifier received from the merchant POS device 130 with the user 101 audio recording.

[0174] From block 1290, the method 280 proceeds to block 295 in Fig. 2. For example, the payment processing system 160 successfully identifies the user 101 based on comparing the received user 101 recording with audio templates from the current customer log. In this example, the payment processing system 160 may continue processing the transaction involving the identified user 101.

[0175] Returning to block 1270, if no audio template from the current customer protocol matches the received voice input, the method 280 proceeds to block 290 in Fig. 2. In another exemplary embodiment, the merchant POS device 130 cannot receive an audio recording of the user 101, or the received audio recording of the user 101 is inappropriate and cannot be used to identify the user 101.

[0176] To Fig. 2, in block 290, the user 101 is identified by the merchant system POS operator 102 via a question and an answer. The method for identifying a user 101 by a merchant point of sale device operator 102 via a question and an answer is described below with reference to the Fig. 13 described procedures 290 are described in more detail.

[0177] Fig. 13 is a block diagram illustrating a method 290 for identifying a user 101 via a question and response by a merchant point-of-sale device operator 102, according to certain exemplary embodiments. The method 290 is described with reference to the Fig. 1 illustrated components.

[0178] At block 1310, the merchant POS device operator 102 issues a question to the user 101. In one example, the merchant POS device operator 102 asks the user 101 for the initials of the user 101. In another example, the merchant POS device operator 102 asks the user 101 for the last four digits of the phone number of the user 101. In another example, the merchant POS device operator 102 asks the user 101 for a configured password. Any suitable question may be issued by the merchant POS device operator 102. In one exemplary embodiment, the answer to the question does not indicate any secure or private information.

[0179] At block 1320, user 101 provides an answer to the question. As indicated in the example questions, the answers may be user 101's initials, the last four digits of user 101's phone number, or a configured password. Any answer to the configured question may be used. In certain embodiments, the answer may be a spoken response, a hand gesture, a keystroke, a display of an ID, or any suitable answer.

[0180] At block 1330, the point-of-sale device operator 102 enters the answer into the merchant point-of-sale device 130. The merchant POS device operator 102 enters the user's 101 answer to the question. In one example, if the user 101 indicates that the user's 101 initials are "AC," the merchant POS device operator 102 enters "AC" into the payment application 133 of the merchant POS device 130. In one example, the user interface 135 of the merchant POS device 130 displays a request for entry of the user's 101 answer. The merchant POS device operator 102 enters the answer via a virtual or physical keyboard, voice dictation, or any suitable manner. In an alternative example, the user 101 enters the answer into the user interface 135 of the merchant POS device 130.

[0181] At block 1340, the merchant point of sale device 130 displays potential users 101 based on the answer to the question. The merchant POS device 130 displays potential users 101 based on the answer to the question. A list of users 101 associated with the answer to the question is displayed to the merchant POS device operator 102 on the merchant POS device 130. For example, if there are ten customers near the merchant beacon device 120, the merchant POS device 130 may have received an answer to the question from the payment processing system 160 associated with each of the respective ten customers. When the merchant POS device 130 receives the input of the answer to the question, only the potential users 101 associated with the answer to the question are displayed to the merchant POS device operator 102.

[0182] In another embodiment, the merchant POS device 130 or payment processing system 160 processing the question presents additional questions until a single matching user 101 remains.

[0183] In the example, if the merchant POS device operator 102 enters "AC" as the initials of the user 101 associated with the transaction, only potential users 101 with those initials are displayed by the payment application 133 to the merchant POS device operator 102. The merchant payment application 133 accesses a database on the payment processing system 160 or other computing device and identifies the initials of the potential users 101 who have provided tokens. The payment application 133 identifies the one or more potential users 101 who have the initials "AC" and displays the identified user 101 accounts to the merchant POS device operator 102. In the example, two of the ten users located near the merchant beacon device 120 have the initials "AC." The user 101 accounts of the two users are displayed to the merchant POS device operator 102.

[0184] In certain exemplary embodiments, all nearby customers who have sent tokens to the merchant POS device 130 are presented to the merchant POS device operator 102, and the merchant POS device operator 102 selects the corresponding user 101 account.

[0185] The payment application 133 may display an image of the potential user 101 accounts presented to the merchant POS device operator 102. For example, each user 101 may associate an image with a user 101 account. When the merchant POS device 130 presents the one or more potential user 101 accounts to the merchant POS device operator 102, the merchant POS device operator 102 may select the appropriate user 101 account based on the image that matches the user 101 conducting the transaction. Other identifying information may be presented instead of or in addition to an image. For example, the name of the user 101 may be displayed, and the merchant POS device operator 102 may identify the potential user 101 by that name. Any appropriate identifying information may be presented.

[0186] At block 1350, the merchant POS device operator 102 selects a user 101 account for use in a transaction. After identifying the displayed image of user 101, the merchant POS device operator 102 may enter a selection of user 101 by operating a user interface 135 control associated with the image or by entering the selection in any suitable manner. If the image does not match any of the potential users, the merchant POS device operator 102 may void the transaction, notify user 101 of the discrepancy, or take any other suitable action.

[0187] In one example, only a single user 101 account is presented in the list of potential users 101. If only a single user 101 account is identified, the method may proceed after the merchant POS device operator 102 verifies that the displayed image matches the user 101. If the image does not match, the merchant POS device operator 102 may void the transaction, notify the user 101 of the discrepancy, or take any other appropriate action.

[0188] From block 1350, the method 290 proceeds to block 295 in Fig. 2 continued.

[0189] Returning to Fig. 2, a transaction is processed at block 295. The method for processing a transaction is described below with reference to the Fig. 14 described procedures 290 are described in more detail.

[0190] Fig. 14 is a block diagram illustrating a method 295 for processing a transaction according to certain exemplary embodiments. The method 295 is described with reference to the Fig. 1 illustrated components.

[0191] At block 1410, the payment processing system 160 sends account information of the identified user 101 to the merchant point-of-sale device 130. The user 101's account with the payment processing system 160 includes, for example, payment account information associated with one or more payment accounts of the user 101. An example payment account of the user 101 includes a credit account, bank account, stored value account, gift card, merchant system credit account, or other relevant user 101 account. In an example embodiment, the merchant point-of-sale device 130 does not receive complete information for one or more of the user 101 payment accounts. For example, the payment information may be garbled or truncated.For example, the merchant POS device 130 may only receive abbreviated payment information sufficient to identify a payment account for a user 101 via the user interface 135.

[0192] At block 1420, the merchant point-of-sale device 130 displays accounts of the identified user 101. In an exemplary embodiment, the merchant POS device 130 does not display complete account information of the user 101 on the user interface 135. For example, for a credit card account, the merchant POS device user interface 135 may display only the name associated with the credit card, the last four digits of the credit card number, and the expiration date.

[0193] At block 1430, the merchant POS device operator 102 selects a user 101 payment account for the transaction and confirms the transaction with user 101's approval. For example, the user 101 views the account options displayed directly on the user interface 135 or listens as the merchant POS device operator 102 reads the user 101 payment account options to the user 101. In an exemplary embodiment, the user 101 selects a payment account option to use in the transaction. For example, the user 101 chooses to use a credit card account. In an exemplary embodiment, the user 101 communicates a selection of a payment account to use in the transaction to the merchant POS device operator 102.In an exemplary embodiment, in response to receiving the user 101 display of the payment account, the merchant POS device operator 102 selects the payment account displayed by the user 101 on the merchant POS device user interface 135.

[0194] At block 1440, the merchant point-of-sale device 130 sends transaction details to the payment processing system 160. For example, the merchant POS device 130 sends transaction details including a transaction total, a merchant system account identifier, and the user's 101 payment option selection. In an exemplary embodiment, the payment processing system 160 receives the transaction details via the network 120. In an exemplary embodiment, the payment processing system 160 extracts payment account information from the user's 101 account corresponding to the received user's 101 payment account selection.

[0195] At block 1450, payment processing system 160 sends a transaction approval request to issuer system 150. In an exemplary embodiment, payment processing system 160 generates a transaction approval request that includes user 101 payment account information, merchant system payment account information, and a total transaction amount. In an exemplary embodiment, payment processing system 160 receives the transaction approval request from payment processing system 160 via network 120.

[0196] At block 1460, the issuer system 150 approves the transaction approval request and sends a transaction authorization approval to the payment processing system 160. In an exemplary embodiment, the issuer system 150 subtracts the current account balance of the user 101 from the credit limit of the user 101 associated with the account with funds of the user 101. In an exemplary embodiment, if the difference between the credit limit and the credit is a positive number greater than the total amount of the transaction, the issuer system 150 approves the transaction authorization. For example, the user 101 would not exceed their credit limit if the transaction were to be approved, and therefore the issuer system 150 allows the transaction approval request.In one exemplary embodiment, payment processing system 160 receives the transaction authorization approval from issuer system 150. In another exemplary embodiment, payment processing system 160 receives a rejection of the transaction approval request in response to issuer system 150 denying the transaction approval request. In one exemplary embodiment, if issuer system 150 approves the transaction approval request, issuer system 150 credits the merchant system account with the amount and adds a debit to the user 101 statement, the credit account statement, the bank statement, or other suitable statement.

[0197] At block 1470, the payment processing system 160 sends a transaction receipt to the merchant point-of-sale device 130. The payment processing system 160 sends, for example, a confirmation of transaction approval or an indication of a declined transaction to the merchant POS device 130 via the network 120. In one exemplary embodiment, the merchant POS device 130 prints a receipt comprising a summary of a successful or unsuccessful transaction via a printing unit that is a component of the merchant POS device 130 or is otherwise communicatively coupled thereto. In another exemplary embodiment, the payment processing system 160 may send a transaction receipt to the user computing device 110 for display on the user computing device.In yet another exemplary embodiment, the merchant POS device 130 sends a transaction receipt to the user computing device 110 to be displayed on the user computing device. Other exemplary embodiments

[0198] Fig.15 illustrates a computing engine 2000 and a module 2050 according to certain example embodiments. The computing engine 2000 may correspond to any of the various computers, servers, mobile devices, embedded systems, or computer systems illustrated herein. The module 2050 may include one or more hardware or software elements configured to assist the computing engine 2000 in performing the various methods and processing functions illustrated herein. The computing engine 2000 may include various internal or mounted components such as a processor 2010, a system bus 2020, a system memory 2030, a storage medium 2040, an input / output interface 2060, and a network interface 2070 for communicating with a network 2080.

[0199] Computing machine 2000 may be implemented as a conventional computer system, an embedded controller, a laptop, a server, a mobile device, a smartphone, a set-top box, a kiosk, a vehicle information system, other processors connected to a television, a custom machine, any other hardware platform, or any combination or plurality thereof. Computing machine 2000 may be a distributed system configured to utilize multiple computing machines interconnected via a data network or bus system.

[0200] Processor 2010 may be configured to execute code or instructions to perform the operations and functionality described herein, manage request flow and address mappings, and perform computations and generate instructions. Processor 2010 may be configured to monitor and control the operation of the components in computing engine 2000. Processor 2010 may be a general-purpose processor, a processor core, a multiprocessor, a reconfigurable processor, a microcontroller, a digital signal processor ("DSP"), an application-specific integrated circuit ("ASIC"), a graphics processing unit ("GPU"), a field-programmable gate array ("FPGA"), a programmable logic device ("PLD"), a controller, a state machine, gated logic, discrete hardware components, any other processing unit, or any combination or plurality thereof.Processor 2010 may be a single processing unit, multiple processing units, a single processor core, multiple processor cores, special-purpose processor cores, coprocessors, or any combination thereof. According to certain embodiments, processor 2010, along with other components of computing engine 2000, may be a virtualized computing engine executing within one or more other computing engines.

[0201] System memory 2030 may include non-volatile memory such as read-only memory ("ROM"), programmable ROM ("PROM"), erasable programmable ROM ("EPROM"), flash memory, or any other device capable of storing program instructions or data with or without applied power. System memory 2030 may also include volatile memory such as random access memory ("RAM"), static random access memory (''SRAM''), dynamic random access memory (''DRAM''), and synchronous dynamic random access memory (''SDRAM''). Other types of RAM may also be used to implement system memory 2030. System memory 2030 may be implemented using a single memory module or multiple memory modules.While system memory 2030 is illustrated as part of computing engine 2000, one skilled in the art will recognize that system memory 2030 may be separate from computing engine 2000 without departing from the scope of the technology of the art. It will also be understood that system memory 2030 may include or operate in conjunction with a non-volatile storage device such as storage medium 2040.

[0202] The storage medium 2040 may comprise a hard disk, a floppy disk, a compact disc read-only memory ("CD-ROM"), a digital versatile disc ("DVD"), a Blu-ray disk, a magnetic tape, a flash memory, another non-volatile storage device, a solid state drive ("SSD"), any magnetic memory, any optical storage device, any electrical storage device, any semiconductor storage device, any physical storage device, any other data storage device, or any combination or plurality thereof. The storage medium 2040 may store one or more operating systems, application programs, and program modules such as module 2050, data, or any other information. The storage medium 2040 may be part of the computing machine 2000 or connected thereto.The storage medium 2040 may also be part of one or more other computing machines that are in communication with the computing machine 2000, such as servers, database servers, cloud storage, network storage, and so on.

[0203] Module 2050 may include one or more hardware or software elements configured to assist computing machine 2000 in performing the various methods and processing functions presented herein. Module 2050 may include one or more sequences of instructions stored as software or firmware in association with system memory 2030, storage medium 2040, or both. Storage medium 2040 may therefore represent examples of machine- or computer-readable media on which instructions or code may be stored for execution by processor 2010. Machine- or computer-readable media may generally refer to any medium or media used to provide instructions to processor 2010. Such machine- or computer-readable media associated with module 2050 may comprise a computer software product.However, it should be understood that a computer software product including module 2050 may also be associated with one or more processes or methods for delivering module 2050 to computing machine 2000 via network 2080, any signal-bearing medium, or any other communication or delivery technology. Module 2050 may also include hardware circuitry or information for configuring hardware circuitry, such as microcode or configuration information for an FPGA or other PLD.

[0204] The input / output ("I / O") interface 2060 may be configured to couple to one or more external devices, receive data from the one or more external devices, and send data to the one or more external devices. Such external devices, along with the various internal devices, are also known as peripherals. The I / O interface 2060 may include both electrical and physical connections to operably couple the various peripherals to the computing engine 2000 or the processor 2010. The I / O interface 2060 may be configured to communicate data, addresses, and control signals between the peripherals, the computing engine 2000, or the processor 2010.The I / O interface 2060 may be configured to implement any standard interface such as a Small Computer System Interface (“SCSI”), SAS interface (“SAS”), Fibre Channel, Peripheral Component Interconnect (“PCI”), PCI Express (PCIe), serial bus, parallel bus, Advanced Technology Attached (“ATA”), Serial ATA (“SATA”), Universal Serial Bus (“USB”), Thunderbolt, FireWire, various video buses, and the like. The I / O interface 2060 may be configured to implement only one interface or bus technology. Alternatively, the I / O interface 2060 may be configured to implement multiple interfaces or bus technologies. The I / O interface 2060 may be configured to operate as part of the system bus 2020, as the entire system bus 2020, or to operate in conjunction therewith.The I / O interface 2060 may include one or more buffers to buffer transfers between one or more external devices, internal devices, the computing engine 2000, or the processor 2010.

[0205] The I / O interface 2060 can couple the computing engine 2000 to various input devices, including mice, touchscreens, scanners, electronic digitizers, sensors, receivers, touchpads, trackballs, cameras, microphones, keyboards, any other pointing devices, or any combination thereof. The I / O interface 2060 can couple the computing engine 2000 to various output devices, including video displays, speakers, printers, projectors, tactile feedback devices, an automation controller, robotic components, actuators, motors, fans, solenoids, valves, pumps, transmitters, signal emitters, lights, and so on.

[0206] Computing engine 2000 may operate in a networked environment using logical connections through network interface 2070 with one or more other systems or computing engines via network 2080. Network 2080 may include wide area networks (WANs), local area networks (LANs), intranets, the Internet, wireless access networks, wired networks, cellular networks, telephone networks, optical networks, or combinations thereof. Network 2080 may be packet-switched, circuit-switched, of any topology, and may use any communication protocol. Communication links within network 2080 may include various digital or analog communication media such as fiber optics, direct optical transmission, waveguides, electrical conductors, wireless links, antennas, radio frequency communications, and so forth.

[0207] The processor 2010 may be connected to the other elements of the computing engine 2000 or the various peripherals described herein through the system bus 2020. It should be understood that the system bus 2020 may be internal to the processor 2010, external to the processor 2010, or both. According to some embodiments, any of the processors 2010, the other elements of the computing engine 2000, or the various peripherals described herein may be integrated into a single device such as a system-on-chip ("SOC"), system-on-package ("SOP"), or an ASIC device.

[0208] In situations where the systems discussed herein collect personal information about users, or may use personal information, users may be provided with a means or option to control whether programs or features collect user information (e.g., information about a user's social network, social actions or activities, occupation, a user preference, or a user's current location), or to control whether and / or how content is received from the content server that may be more relevant to the user. In addition, certain data may be treated in one or more ways before it is stored or used so that personally identifiable information is removed.For example, a user's identity may be treated so that no personally identifiable information can be determined for the user, or a user's location may be generalized, extracting location information (such as a city, ZIP code, or state) so that a specific user location cannot be determined. Thus, the user may have control over the extent to which information about the user is collected and used by a content server.

[0209] Embodiments may include a computer program embodying the functions described and illustrated herein, the computer program implemented in a computer system comprising instructions stored on a machine-readable medium and a processor that executes the instructions. However, it should be appreciated that there could be many different ways to implement embodiments in computer programming, and the embodiments should not be construed as limited to any one set of computer program instructions. A skilled programmer would also be able to write such a computer program to implement an embodiment of the disclosed embodiments based on the attached flowcharts and associated description in the application text.Therefore, disclosure of a specific series of program code instructions is not considered necessary for a proper understanding of how to make and use embodiments. Further, those skilled in the art will recognize that one or more aspects of the embodiments described herein may be carried out by hardware, software, or a combination thereof, as may be embodied in one or more computer systems. Furthermore, any reference to an act performed by a computer should not be construed as being performed by a single computer, as more than one computer may perform the act.

[0210] The exemplary embodiments described herein may be used with computer hardware and software that perform the methods and processing functions described herein. The systems, methods, and procedures described herein may be embodied in a programmable computer, computer-executable software, or digital circuitry. The software may be stored on computer-readable media. Computer-readable media may include, for example, a floppy disk, RAM, ROM, hard disk, removable media, flash memory, memory stick, optical media, magneto-optical media, CD-ROM, etc. Digital circuitry may include integrated circuits, gate arrays, device logic, field-programmable gate arrays (FPGA), etc.

[0211] The exemplary systems, methods, and acts described in the previously presented embodiments are illustrative, and in alternative embodiments, certain acts may be performed in a different order, in parallel with one another, omitted entirely, and / or combined between different exemplary embodiments and / or certain additional acts may be performed without departing from the scope and spirit of the various embodiments. Accordingly, such alternative embodiments are included within the scope of the following claims, which are to be given the broadest interpretation so as to encompass such alternative embodiments.

[0212] Although specific embodiments have been described in detail above, the description is for illustrative purposes only. It should therefore be understood that many aspects described above are not intended to be required or essential components unless expressly stated otherwise. Modifications of and equivalent components or acts corresponding to the disclosed aspects of the exemplary embodiments, in addition to those described above, may be practiced by one of ordinary skill in the art with the benefit of the present disclosure without departing from the spirit and scope of embodiments defined in the following claims, the scope of which is to be given the broadest interpretation to encompass such modifications and equivalent structures.

Claims

[1] A computer-implemented method for identifying users at locations by comparing facial images of users (101) with facial templates of users (101), comprising: Transmitting an identifier by a computing device that is associated with the computing device at a location; Receiving, by the computing device and from one or more other computing devices, one or more facial templates, each facial template associated with a corresponding user computing device (110) that received the transmitted identifier at the location and further transmitted the identifier to the one or more computing devices, each facial template comprising a representation of the corresponding facial image of the corresponding user; Capturing, by a camera module of the computing device, a video feed of an environment external to the computing device; extracting, by the computing device, a facial image of a user (101) from the video feed; generating, by the computing device, a face template of the user (101) based on the extracted face image of the user (101), wherein the face template of the user (101) is a representation of the face image of the user (101); Retrieving, by the computing device, one or more stored facial templates; determining, by the computing device, a similarity between the generated facial template of the user (101) and each of the one or more stored facial templates; identifying, by the computing device, a particular stored face template corresponding to the generated face template of the user (101) based on the similarity between the generated face template of the user (101) and the particular stored face template exceeding a threshold; Sending, by the computing device, a response to a request to identify a user (101) comprising an indication of the identified user (101). [2] The method of claim 1, further comprising receiving, by the computing device, a request to identify a user (101). [3] The method of claim 1, wherein the environment external to the computing device comprises the user (101) being in proximity to one or more merchant point-of-sale (POS) devices connected to the computing device. [4] The method of claim 2, wherein the request to identify the user (101) is received from a merchant POS device (130) connected to the computing device, and wherein the response to the request to identify a user (101) is sent to the merchant POS device (130). [5] The method of claim 4, further comprising processing, by the merchant POS device (130), a transaction using payment account information associated with the identified user (101). [6] The method of claim 3, further comprising: Determining, by the computing device and based on a frame of the video feed, a position of a face of the user (101) and positions of each of the one or more merchant POS devices at a merchant location; and Assigning, by the computing device and based on distances between the face of the user (101) and each of the one or more merchant POS devices in the frame of the video feed, the user (101) to a particular merchant POS device of the one or more merchant POS devices, wherein the indication of the identity of the user (101) further comprises an identifier of the particular merchant POS device and an indication of the assignment to the user (101) of the particular merchant POS device. [7] A computer-implemented method for identifying users (101) at locations by comparing facial images of users with facial templates of users (101), comprising: Receiving, by one or more computing devices and from a user computing device (110), an account identifier of a user's account (101) associated with the user computing device (110) and a merchant beacon identifier, wherein the user computing device (110) transmits the merchant beacon identifier received from a merchant beacon device located at the site of a merchant system over a wireless medium; retrieving, by the one or more computing devices, a face template associated with the user's account (101) based on the account identifier of the account associated with the user (101); Identifying, by the one or more computing devices, the location of the merchant system associated with the merchant beacon identifier; adding, by the one or more computing devices, the facial template associated with the user's account (101) and the associated merchant system location to a current customer log comprising facial templates corresponding to user computing devices transmitting merchant beacon identifiers received from merchant systems at the merchant system location associated with the merchant beacon identifier; Sending, by the one or more computing devices and to a merchant computing device at the location of the merchant system, the current customer log comprising the facial template associated with the user's (101) account, wherein the merchant computing device identifies the user (101) based on identifying the facial template associated with the user's (101) account from the current customer log that is similar to a facial template of the user (101) generated from a facial image of the user (101) captured by the computing device [8] The method of claim 7, further comprising: Determining, by the one or more computing devices, that the user computing device (110) no longer maintains a network connection with the merchant beacon device; removing, by the one or more computing devices, the facial template associated with the user's account (101) and the associated location of the merchant system from the current customer log if it has been determined that the user computing device (110) no longer maintains a network connection with the merchant beacon device to update the current customer log, and Transmission of the updated customer log by the one or more computing devices to the computing device. [9] The method of claim 8, wherein determining that the user computing device (110) no longer maintains a network connection with the merchant beacon device comprises detecting no subsequent transmission from the user computing device (110) of the user account identifier and the merchant beacon identifier within a threshold time period. [10] The method of claim 7, further comprising: Receiving, by the one or more computing devices and from the computing device, a facial template associated with an identified user (101), a transaction amount, and a merchant account identifier from the merchant computing device at the merchant system location; Determining, based on the received face template associated with an identified user (101), an account identifier associated with the identified user (101); Sending, by the one or more computing devices and to a point of sale device, payment account information associated with a user account associated with the account identifier associated with the identified user; Receiving, by the one or more computing devices and from the point of sale device, an indication of a selection of a particular payment account information from the payment account information for use in a transaction; generating, by the one or more computing devices, a transaction approval request comprising the transaction amount, the merchant account identifier, and the particular payment account information element; Sending, by the one or more computing devices, the transaction approval request to an issuer system associated with the particular payment account information; Receiving, by the one or more computing devices and from the issuer system, data comprising an authorization of the transaction approval request; and Sending, by the one or more computing devices and to the point of sale device for display on the point of sale device, a message regarding the approval of the payment authorization request. [11] The method of claim 10, further comprising receiving, from the computing device, an assignment of the identified user (101) to the point-of-sale device at the merchant system location, wherein the payment account information is sent to the payment device based on the assignment. [12] Computer program product comprising: a non-transitory, computer-readable medium embodying computer-executable program instructions that, when executed by a computer, cause the computer to identify users (101) at locations by comparing facial images of users (101) with facial templates of users (101), the computer-executable program instructions comprising: computer-readable program instructions for transmitting an identifier by a computing device that is connected to the computing device at a location; computer-readable program instructions for receiving, from one or more other computing devices, one or more facial templates, each facial template being associated with a user computing device (110) that receives the transmitted identifier at the location and further transmits the identifier to the one or more computing devices, each facial template comprising a representation of the corresponding facial image of the corresponding user (101); computer-readable program instructions for receiving a request to identify a user (101); computer-readable program instructions for capturing, by a camera module, a video feed of an environment external to the computer; computer-readable program instructions for extracting a facial image of a user (101) from the video feed; computer-readable program instructions for generating a facial template of the user (101) based on the facial image, wherein the facial template of the user (101) is a representation of the facial image; computer-readable program instructions for retrieving one or more stored facial templates associated with one or more corresponding user computing devices that sent the transmitted identifier; computer-readable program instructions for determining a similarity between the generated facial template of the user (101) and each of the one or more facial templates; computer-readable program instructions for identifying a particular stored facial template corresponding to the generated facial template of the user (101) based on the similarity between the generated facial template of the user (101) and the particular stored facial template exceeding a threshold; and computer-readable program instructions for sending a response to a request to identify the user (101) comprising an indication of an identity of the user (101). [13] The computer program product of claim 12, wherein the environment external to the computer comprises the user (101) being in proximity to one or more merchant point-of-sale (POS) devices. [14] The computer program product of claim 12, wherein the request to identify a user (101) is received from a merchant POS device (130), and wherein the response to the request to identify a user (101) is sent to the merchant POS device (130). [15] The computer program product of claim 13, wherein the merchant POS device (130) processes a transaction using payment account information associated with the identified user (101) at a time the response to the request to identify the user (101) is sent to the merchant POS device (130). [16] The computer program product of claim 13, further comprising: computer readable program instructions for determining, based on a frame of the video feed, a position of a face of the user (101) and positions of each of the one or more merchant POS devices at a merchant location; computer-readable program instructions for assigning, based on distances between the face of the user (101) and each of the one or more merchant POS devices in the frame of the video feed, the user (101) to a particular merchant POS device of the one or more merchant POS devices, wherein the indication of the identity of the user (101) further comprises an identifier of the particular merchant POS device and an indication of the assignment of the user (101) to the particular merchant POS device. [17] A system for identifying users (101) at locations by comparing facial images of users (101) with facial templates of users (101), comprising: a storage device; and a processor communicatively coupled to the storage device, the processor executing application code instructions stored in the storage device to cause the system to: Receiving, from a user computing device (110), an account identifier of a user's account (101) associated with the user computing device (110) and a merchant beacon identifier, wherein the user computing device (110) transmits the merchant beacon identifier received via a network connection from a merchant beacon device located at the site of a merchant system; Retrieving a face template associated with the user's account (101) based on the account identifier of the user's account (101); Identifying the location of the merchant system associated with the merchant beacon identifier; Adding the facial template associated with the user's account (101) and the associated merchant system location to a current customer log comprising facial templates of users (101) corresponding to user computing devices that transmit merchant beacon identifiers received from merchant systems over the network connection at the merchant system location associated with the merchant beacon identifier; and Sending, to a computing device at the merchant system location, the current customer log comprising the facial template associated with the user's account (101), wherein the computing device identifies the user (101) based on identifying the facial template associated with the user's account from the current customer log, which is similar to a facial template of the user (101) generated from a facial image of the user (101) captured by the computing device. [18] The system of claim 17, wherein the processor is further configured to execute computer-readable program instructions stored on the storage device to cause the system to: Determining that the user computing device (110) no longer maintains a network connection with the merchant beacon device; and Removing the facial template associated with the user's account (101) and the merchant system location from the current customer log when it is determined that the user computing device (110) no longer maintains a network connection with the merchant beacon device to update the current customer log. [19] The system of claim 17, wherein determining that the user computing device (110) no longer maintains a network connection to the merchant beacon device comprises detecting that no subsequent transmission is received from the user computing device (110) of the user account identifier and the merchant beacon identifier within a threshold time period. [20] The system of claim 17, wherein the processor is further configured to execute computer-readable program instructions stored on the storage device to cause the system to: Receiving, by the computing device, a facial template associated with an identified user (101), a transaction amount, and a merchant account identifier from the computing device at the merchant system location; determining, based on the received facial template associated with the identified user (101), an account identifier associated with the identified user; Sending, to a point of sale device, payment account information associated with a user account associated with an account identifier of the identified user (101); Receiving, from the point of sale device, an indication of a selection of a particular payment account information from the payment account information for use in a transaction; Generating a transaction approval request including the transaction amount, the merchant account identifier, and the specific payment account information; Sending the transaction approval request to an issuer system associated with the specific payment account information; Receiving, from the issuer system, data that includes approval of the transaction approval request; and Sending, to the point of sale device for display on the point of sale device, a receipt including a note of approval of the transaction authorization request.

Images