Recognizing a consumer in a transaction via facial recognition

Abstract

The merchant and user registration payment processing system uses facial templates based on user images. Users log into the payment application via their computing devices, which receive identifiers from the merchant beacon device and transmit them to the payment processing system. The payment processing system then transmits the facial templates to merchant camera devices used by other users also logged into the payment application within range of the merchant beacon device. The merchant camera devices compare captured facial images with the received facial templates to identify the user. The merchant POS operator selects the user's account. The merchant POS device transmits transaction details to the payment processing system, which processes the transaction using the issuer system. The payment processing system receives approval for the transaction authorization request and transmits a receipt to the merchant POS device.

Description

[0001] Case Analysis

[0002] This application is a divisional application of Chinese invention patent application 201680020048.6, filed on April 28, 2016. Technical Field

[0003] This disclosure relates to improving user convenience in transactions by identifying the user account used in a transaction through facial recognition of the user via a payment processing system. Background Technology

[0004] When consumers make purchases at a merchant's location, many methods of transaction are available. Consumers can use a variety of different cards or accounts to make purchases, such as gift cards, debit cards, credit cards, stored-value cards, and other cards or accounts. The user account identifier and other data represented by the card can be communicated to the merchant system via magnetic stripe, near-field communication technology involving the user's computing device, and other appropriate mechanisms.

[0005] The current application for transactions at merchant locations does not offer consumers the opportunity to conduct hands-free transactions. Furthermore, the current application requires consumers to perform actions that identify them by providing a user account identifier or other data to the merchant system. Summary of the Invention

[0006] This document provides a computer-implemented method for hands-free transactions using user facial recognition. In an example embodiment, a merchant registers with a payment processing system. A user creates an account with the payment processing system and transmits their image to the system to create a facial template associated with their account. The user logs into a payment application via their computing device and enters their merchant system location. The user computing device receives a merchant beacon device identifier from a merchant beacon device and transmits it to the payment processing system. The payment processing system transmits the facial template to a merchant camera device corresponding to other users logged into the payment application and within the network range of the merchant beacon device. The merchant camera device captures an image of the user's face and identifies the user by comparing the captured image with the received facial template. A merchant point-of-sale (POS) operator selects the user's account from one or more displayed accounts for the transaction. The POS transmits transaction details to the payment processing system, which generates a transaction authorization request to be transmitted to the issuer system associated with the selected user account for the transaction. The payment processing system receives approval for the transaction authorization request and transmits a receipt to the POS.

[0007] In some other examples described herein, systems and computer program products are provided that enable hands-free transactions using a user's facial recognition.

[0008] These and other aspects, objects, features, and advantages of the exemplary embodiments will become apparent to those skilled in the art upon taking into consideration the following detailed description of the exemplary embodiments shown. Attached Figure Description

[0009] Figure 1 This is a block diagram illustrating a system for hands-free transactions using a user's facial recognition, according to some example embodiments.

[0010] Figure 2 This is a flowchart illustrating a method for conducting hands-free transactions using a user's facial recognition according to certain example embodiments.

[0011] Figure 3 This is a flowchart illustrating a method for a merchant system to register with a payment processing system and install hardware at the merchant system location, according to certain example embodiments.

[0012] Figure 4 This is a flowchart illustrating a method for a user to register an account with a payment processing system, according to certain example embodiments.

[0013] Figure 5 This is a flowchart illustrating a method for creating a facial template associated with a user account, according to certain example embodiments.

[0014] Figure 6 This is a flowchart illustrating a method for creating an audio template associated with a user account, according to certain example embodiments.

[0015] Figure 7 This is a flowchart illustrating a method by which a user computing device receives a merchant beacon identifier broadcast by a merchant beacon device, according to certain example embodiments.

[0016] Figure 8 This is a flowchart illustrating a method by which a camera device receives a facial template of each user within the range of a merchant beacon device, according to certain example embodiments.

[0017] Figure 9 This is a flowchart illustrating a method by which a camera device receives a notification from a payment processing system when a user enters or leaves the network range of a merchant beacon device, according to certain example embodiments.

[0018] Figure 10 This is a flowchart illustrating a method by which a user initiates a transaction at a merchant's point-of-sale device, according to certain example embodiments.

[0019] Figure 11 This is a flowchart illustrating a method for a camera device to identify a user via facial recognition, according to certain example embodiments.

[0020] Figure 12 This is a flowchart illustrating a method for a payment processing system to identify a user via voice recognition, according to certain example embodiments.

[0021] Figure 13 This is a flowchart illustrating a method by which a merchant's point-of-sale equipment operator identifies a user via challenges and responses, according to certain example embodiments.

[0022] Figure 14 This is a flowchart illustrating a method for conducting a transaction according to certain example embodiments.

[0023] Figure 15 This is a block diagram illustrating a computing machine and module according to certain example embodiments. Detailed Implementation

[0024] The example embodiments described herein provide computer-implemented techniques for conducting hands-free transactions using a user's facial recognition.

[0025] In an example embodiment, a merchant registers with a payment processing system. The merchant system installs one or more merchant beacon devices, one or more merchant point-of-sale devices, and one or more merchant camera devices at the merchant system location. For example, a user creates an account with the payment processing system and downloads a payment application on a user computing device associated with the user. In one example, the user transmits their image and / or audio recording to the payment processing system to create a facial template and / or audio template associated with the user account. The user enters the merchant system location and logs into the payment application via the user computing device. The user computing device receives a merchant beacon device identifier from the merchant beacon device and transmits the identifier to the payment processing system. The payment processing system transmits the facial template to the merchant camera device corresponding to the user's user computing device within the network range of the merchant beacon device and the user logged into the payment application. The merchant camera device captures the user's facial image and identifies the user by comparing the captured facial image with the received facial template. Alternatively, the user submits an audio recording to the payment processing system via a merchant point-of-sale device, which identifies the user by comparing the received audio recording with the user's audio template. In another embodiment, the merchant point-of-sale (POS) operator identifies the user based on the user's response to a challenge. After user identification, the POS displays one or more of the user's accounts. The POS operator selects the user's account to be used in the transaction. The POS transmits the transaction details to a payment processing system, which generates a transaction authorization request to be sent to the issuer system associated with the user account selected for the transaction. The payment processing system receives approval for the transaction authorization request and sends a receipt to the POS.

[0026] In an example embodiment, the merchant system registers with the payment processing system. The merchant system operator installs a payment application at the merchant point-of-sale device. In another example, the merchant system operator installs the payment application on multiple merchant point-of-sale devices at the merchant system location. Merchant beacon devices receive beacon identification codes from the payment processing system. For example, the merchant system operator installs one or more merchant beacon devices at the merchant system location. The merchant beacon devices broadcast the merchant beacon identification codes via wireless communication at the merchant system location. The merchant system operator installs merchant camera devices at merchant system locations corresponding to the merchant point-of-sale devices. In another example, multiple merchant camera devices are installed at the merchant system location, each corresponding to a specific merchant point-of-sale device. In yet another example, a specific merchant camera device may correspond to two or more specific merchant point-of-sale devices. The payment processing system receives the camera device identifier and associates it with the corresponding beacon identifier of the merchant beacon device.

[0027] In an example embodiment, a user registers with a payment processing system. For example, the user accesses the payment processing system via a user computing device associated with the user. The user registers with the payment processing system and downloads a payment application to the user computing device. In an example embodiment, the payment processing system creates a facial template associated with the user's account. For example, the payment application displays a request to capture a facial image to the user via the user computing device. The user selects the option to capture a facial image. The payment application activates the camera module on the user computing device, and the user captures their own facial image. The payment processing system receives the facial image. Based on the received facial image, the payment processing system creates a facial template associated with the user's account. The payment processing system deletes the received facial image. In another example embodiment, the payment processing system creates an audio template associated with the user's account. The payment application requests and receives user audio via the user computing device. Based on the received audio of the user's voice, the payment application creates an audio template associated with the user's account. The payment processing system deletes the received audio of the user's voice.

[0028] The user logs into the payment application on their computing device. The user carries the computing device within a threshold distance of the merchant beacon device at the merchant system's location. The computing device receives the merchant beacon identifier broadcast by the merchant beacon device and transmits the received merchant beacon identifier and user account identifier to the payment processing system. The payment processing system receives the merchant beacon identifier and user account identifier. The payment processing system extracts the facial template associated with the user account identifier and identifies the merchant camera device associated with the merchant beacon device identifier.

[0029] The payment processing system transmits the facial template of the identified user to the merchant camera device associated with the merchant beacon device identifier. For example, a facial template associated with an identified user account is transmitted to the merchant camera device. The merchant camera device receives the user's facial template. The merchant camera device adds the user's facial template to the current consumer log. The merchant camera device periodically updates the current consumer log based on updates received from the payment processing system. For example, the payment processing system transmits subsequent facial templates of subsequent users with the threshold distance required for the user's computing device to establish a wireless network connection to the merchant beacon device, via which the user logs into the payment application. In this example, the payment processing system receives the merchant beacon device identifier transmitted by the user's computing device and transmits the subsequent user's facial template to the merchant camera device. In another example, in response to detecting that the user's computing device associated with a user in the current consumer log is no longer maintaining a network connection with the merchant beacon device, no longer observing the merchant beacon device, or no longer logged into the payment application, the payment processing system transmits a notification that the user has left the merchant location to the merchant camera device. In this example, the merchant camera device deletes the indicated user from the current consumer log.

[0030] A user approaches a merchant's point-of-sale (POS) device. The POS device operator tallies the items the user intends to purchase. The POS device operator prompts the user to select a payment option. The user instructs the POS device operator to initiate the transaction via a payment application. For example, as described earlier, the payment application is installed on both the POS device and the user's computing device. The POS device operator selects an option on the POS device to initiate the transaction using the payment application. The POS device, via a payment processing system, transmits a request to identify the user to the merchant's camera device. In this example, the payment processing system communicates with both the merchant's camera device and the POS device.

[0031] In another example embodiment, the merchant point-of-sale device receives an instruction from the merchant camera device that has already identified the user, and the merchant point-of-sale device displays an option to the user to initiate a transaction using a payment application. For example, the merchant point-of-sale device may receive an instruction that has already been identified by the merchant camera device directly over the network. In another example, the merchant camera device transmits the instruction that has identified the user to the payment processing system, and then the payment processing system transmits the instruction that has identified the user to the merchant point-of-sale device. In these example embodiments, the merchant point-of-sale device operator, in response to the merchant point-of-sale device displaying the option to initiate a transaction using a payment application, asks the user if they wish to initiate a transaction using that payment application. In these example embodiments, the user instructs the merchant point-of-sale device operator to initiate a transaction via the payment application, and the merchant point-of-sale device operator selects the corresponding option on the merchant point-of-sale device's user interface.

[0032] The merchant camera device captures video of the user. For example, the user is located in front of the point-of-sale device and the merchant camera device is positioned to capture video of the user's face. In an example embodiment, the merchant camera device only begins capturing video of the user when it receives a request to identify the user. For example, the merchant camera device receives a request to identify the user from a payment processing system or from the merchant's point-of-sale device. In another example embodiment, the merchant camera device begins capturing video when it receives from the payment processing system that a user computing device associated with the user has established a network connection with the merchant beacon device. In this example embodiment, the merchant camera device does not capture video when there is no user with an associated user computing device within the network range of the merchant beacon device. The merchant camera device extracts the user's facial image from the captured video and generates a facial template from the captured facial image. The merchant camera device deletes the captured video and the extracted facial image. The merchant camera device retrieves the facial template from the current consumer log. For example, the current consumer log includes a list of users and associated facial templates of users associated with user computing devices that have established a network connection with the merchant beacon device at the current merchant system location. In an example embodiment, the current consumer log includes volatile or temporary storage. For example, the current consumer log is not saved, and user information is added or deleted from the current consumer log when the user's computing device associated with the corresponding user enters or leaves the network range of the merchant beacon device. The merchant camera device compares a facial template generated from the extracted facial image with a facial template from the current consumer log. If a match exists between the facial template from the current consumer log and the generated facial template, the merchant camera device can identify the user. If no match exists between the facial template from the current consumer log and the generated facial template, the merchant camera device cannot identify the user. If the merchant camera device can identify the user, it notifies the payment processing system of the user's identity. In another example embodiment, if the merchant camera device can identify the user, it notifies the merchant point-of-sale device of the user's identity.

[0033] If the merchant camera device cannot identify a user based on facial recognition, it notifies the payment processing system that it cannot identify the user based on facial recognition. In an example embodiment, if facial recognition fails to identify the user, the payment processing system identifies the user based on voice recognition. In another example embodiment, the payment processing system does not identify the user based on voice recognition. In an example embodiment, if the payment processing system identifies the user based on voice recognition, it retrieves an audio template corresponding to the user from the current consumer log. The payment processing system sends a request to the merchant system's point-of-sale device to record the user's audio. The merchant system's point-of-sale device displays the audio recording request to the user, records the user's voice input, and sends the voice input to the payment processing system. The payment processing system compares the received voice input with the audio template corresponding to the user retrieved from the current consumer log. If there is a match between the audio template from the current consumer log and the received user's voice input, the payment processing system can identify the user. If there is no match between the audio template from the current consumer log and the received user's voice input, the payment processing system cannot identify the user.

[0034] If the payment processing system cannot identify a user based on voice recognition, it notifies the merchant point-of-sale (POS) operator to issue a challenge to the user. The user provides a challenge response, and the POS operator inputs the response into the POS device. The POS device displays potential users based on the challenge response. For example, the POS device transmits the response to the payment processing system, and the payment processing system accesses a database containing lists and tables that associate challenges with corresponding responses and users. In this example, the payment processing system identifies the user by associating the challenge and response to identify one or more users in the database. In this example, the payment processing system transmits one or more identified users to the POS device. In this example, the POS device displays one or more identified users to the POS operator. The POS operator selects a user. In this example, the POS operator can compare a visual image or name of the user displayed on the user's computing device with the visual appearance of the current consumer at the POS device or a document presented to the POS operator by the user. In this example, the POS device transmits the identity of the user identified by the POS operator.

[0035] The point-of-sale (POS) device displays the identified user's account. For example, after receiving the user's identity from the merchant's camera device, the merchant's POS device, or after identifying the user, the payment processing system transmits information associated with one or more accounts belonging to the identified user. The merchant POS device operator selects the user account for the transaction and, with the user's permission, confirms the transaction. The merchant POS device transmits the transaction details to the payment processing system. For example, transaction details may include the total transaction amount, the user account selected for use in the transaction, the merchant's account used in the transaction, and other useful or relevant information. The payment processing system transmits the transaction authorization request to the issuer system. For example, the issuer system is associated with the user account selected for use in the transaction. The issuer system approves the transaction authorization request and transmits the transaction authorization approval to the payment processing system. The payment processing system transmits the transaction receipt to the merchant POS device.

[0036] By using and relying on the methods and systems described herein, payment processing systems, merchant camera devices, merchant beacon devices, user computing devices, and merchant point-of-sale devices enable users to transact with merchant systems without requiring them to interact with user computing devices or generate identity documents or physical payment cards, as is required in some current technologies. Similarly, the systems and methods described herein can reduce the input required from users via user computing devices and the input required for merchant point-of-sale device operators to identify users.

[0037] Example System Architecture

[0038] Turning now to the accompanying drawings, in which the same numbers in all the drawings indicate similar (but not necessarily identical) elements, an example embodiment is described in detail.

[0039] Figure 1 This is a block diagram of a system 100 for conducting hands-free transactions using facial recognition of user 101, according to some example embodiments. For example... Figure 1 As shown, system 100 includes a user computing device 110, a merchant point-of-sale device 130, a merchant camera device 140, an issuer system 150, and a payment processing system 160, all configured to communicate with each other via one or more networks 170. In some embodiments, a user associated with the device must install an application and / or perform feature selection to obtain the benefits of the technology described herein.

[0040] In the example embodiment, network 170 includes a local area network (“LAN”), a wide area network (“WAN”), an intranet, the Internet, a storage area network (“SAN”), a personal area network (“PAN”), a metropolitan area network (“MAN”), a wireless local area network (“WLAN”), a virtual private network (“VPN”), a cellular or other mobile communication network, Bluetooth, Bluetooth Low Energy, NFC, or any combination thereof, or any other suitable architecture or system that facilitates the communication of signals, data, and / or messages. Throughout the discussion of the example embodiment, it should be understood that the terms “data” and “information” are used interchangeably herein to refer to text, images, audio, video, or any other form of information that can exist in a computer-based environment.

[0041] Each of the user computing device 110, merchant point-of-sale device 130, merchant camera device 140, issuer system 150, and payment processing system 160 includes a communication module capable of transmitting and receiving data via network 170. For example, each of the user computing device 110, merchant point-of-sale device 130, merchant camera device 140, issuer system 150, and payment processing system 160 may include a server, desktop computer, laptop computer, tablet computer, television with one or more processors built into and / or coupled thereto, smartphone, handheld computer, personal digital assistant (“PDA”), or any other wired or wireless processor-driven device. Figure 1 In the example embodiment shown, the user computing device 110, the merchant point-of-sale device 130, the merchant camera device 140, the issuer system 150, and the payment processing system 160 are operated by the user 101, the merchant beacon device 120 operator, the merchant point-of-sale (“POS”) device 130 operator, the merchant camera device 140 operator, the issuer system 150 operator, and the payment processing system 160, respectively.

[0042] Example user computing device 110 includes an antenna 111, a Wi-Fi controller 112, a payment application 113, a user interface 115, a data storage unit 116, a camera module 117, a web browser 118, and a communication application 119.

[0043] In an example embodiment, antenna 111 is a means of communication between user computing device 110 and merchant beacon device 120. In an example embodiment, Wi-Fi controller 112 outputs radio signals or listens for radio signals from merchant beacon device 120 via antenna 111. In another example embodiment, a Bluetooth controller or a Near Field Communication (“NFC”) controller is used. In an example embodiment, Wi-Fi controller 112 outputs radio signals or listens for radio signals from payment card device 120 via antenna 111.

[0044] In an example embodiment, the Wi-Fi controller 112 is capable of sending and receiving data, performing authentication and encryption functions according to a Wi-Fi-specified procedure, and instructing the user computing device 110 on how to listen for transmissions from the merchant beacon device 120 or configure the user computing device 110 into various power-saving modes. In another example embodiment, the user computing device 110 includes a Bluetooth controller or an NFC controller capable of performing similar functions. The example Wi-Fi controller 112 communicates with the payment application 113 and is capable of sending and receiving data via a wireless, Wi-Fi communication channel. In another example embodiment, the Bluetooth controller 112 or the NFC controller 112 uses the Bluetooth or NFC protocol to perform functions similar to those of the Wi-Fi controller 112. In an example embodiment, the Wi-Fi controller 112 activates the antenna 111 to create a wireless communication channel between the user computing device 110 and the merchant beacon device 120. The user computing device 110 communicates with the merchant beacon device 120 via the antenna 111. In an example embodiment, when the user computing device 110 has been activated, the Wi-Fi controller 112 polls radio signals via antenna 111 or listens for radio signals from merchant beacon device 120.

[0045] In an example embodiment, payment application 113 is a program, function, routine, applet, or similar entity that exists on and operates on user computing device 110. In some example embodiments, user 101 must install payment application 113 and / or make feature selections on user computing device 110 to obtain the benefits of the technology described herein. In an example embodiment, user 101 can access payment application 113 on user computing device 110 via user interface 115. In an example embodiment, payment application 113 may be associated with payment processing system 160. In another example embodiment, payment application 113 may be associated with a merchant system associated with merchant beacon device 120, merchant point-of-sale device 130, and merchant camera device 140. In yet another example embodiment, there are two payment applications 113, one associated with the merchant system and the other associated with payment processing system 160.

[0046] In an example embodiment, user interface 115 enables user 101 to interact with payment application 113 and / or web browser 118. For example, user interface 115 may be a touchscreen, a voice-based interface, or any other interface that allows the user to provide input and receive output from an application or module on user computing device 110. In an example embodiment, user 101 interacts with payment application 113 and / or web browser 118 via user interface 115 to configure user 101's account on payment processing system 160. In another example embodiment, user 101 interacts with payment application 113 and / or web browser 118 via user interface 115 to enable hands-free payment as needed.

[0047] In an example embodiment, data storage unit 116 includes a local or remote data storage structure suitable for storing information and accessible to user computing device 110. In an example embodiment, data storage unit 116 stores encrypted information, such as HTML5 local storage.

[0048] In the example embodiment, camera module 117 can be any module or function of user computing device 110 that acquires digital images. Camera module 117 resides on user computing device 110 or is logically connected to user computing device 110 in any way. For example, camera module 117 is connected to user computing device 110 via network 170. Camera module 117 is capable of acquiring personal images or video scans. Any other suitable image capture device can be represented by camera module 117.

[0049] In an example embodiment, user 101 can use communication application 119, such as web browser application 118 or a standalone application, to view, download, upload or otherwise access documents or web pages via distributed network 170.

[0050] In an example embodiment, web browser 118 enables user 101 to interact with web pages using user computing device 110. In an example embodiment, user 101 can access an account maintained by payment processing system 160 via web browser 118. In another example embodiment, user 101 can access a merchant system website via web browser 118. In some example embodiments described herein, one or more functions performed by payment application 113 can also be performed by the web browser 118 application associated with payment processing system 160.

[0051] In an example embodiment, communication application 119 can interact with a web server or other computing device (including user computing device 110 and the web server of the merchant system) connected to network 170.

[0052] In some example embodiments, one or more functions as described herein performed by payment application 113 can also be performed by web browser application 118, such as a web browser application associated with a merchant system website or with payment processing system 160. In some example embodiments, one or more functions as described herein performed by payment application 113 can also be operated by user computing device 110. In some example embodiments, one or more functions as described herein performed via web browser 118 can also be performed via payment application 113.

[0053] Example merchant beacon device 120 includes an antenna 121 and a Wi-Fi controller 122. In an example embodiment, a merchant system location includes one or more merchant beacon devices 120 installed at the merchant system location. In an example embodiment, each installed merchant beacon device 120 is associated by a payment processing system 160 with a specific merchant camera device 140 installed at the merchant location. For example, the payment processing system 160 may include a database that associates merchant beacon device 120 identifiers with merchant camera device 140 identifiers used to associate the merchant camera device 140. For example, the merchant camera device 140 identifier may include a hardware identifier specific to the device, such as a serial number or MACID. In another example, the merchant beacon device 120 identifier may include a hardware identifier specific to the beacon device or an identifier generated by the payment processing system 160 and stored in the merchant beacon device 120. Example merchant beacon device 120 is programmed to broadcast, transmit, or otherwise transmit a specific merchant beacon device 120 identifier to any user computing device 110 within a threshold distance required to maintain wireless network 170. For example, wireless network 170 may include Wi-Fi network 170, Bluetooth network 170, NFC network 170, or any other suitable wireless network 170.

[0054] In an example embodiment, antenna 121 is a communication device between user computing device 110 and merchant beacon device 120. In an example embodiment, Wi-Fi controller 122 outputs radio signals or listens for radio signals from user computing device 110 via antenna 121. In another example embodiment, a Bluetooth controller or a Near Field Communication (“NFC”) controller is used. In an example embodiment, Wi-Fi controller 122 outputs radio signals or listens for radio signals from payment card device 120 via antenna 121.

[0055] In an example embodiment, the Wi-Fi controller 122 is capable of sending and receiving data, performing authentication and encryption functions, and instructing the merchant beacon device 120 on how to listen for transmissions from the user computing device 110 or configuring the merchant beacon device 120 into various power-saving modes, according to Wi-Fi-specific procedures. In another example embodiment, the merchant beacon device 120 includes a Bluetooth controller or an NFC controller capable of performing similar functions. The example Wi-Fi controller 122 communicates with the payment application 113 and is capable of sending and receiving data via a wireless Wi-Fi communication channel. In another example embodiment, the Bluetooth controller 122 or the NFC controller 122 uses the Bluetooth or NFC protocol to perform functions similar to those of the Wi-Fi controller 122. In an example embodiment, the Wi-Fi controller 122 activates the antenna 121 to create a wireless communication channel between the user computing device 110 and the merchant beacon device 120. The merchant beacon device 120 communicates with the user computing device 110 via the antenna 121. In an example embodiment, when the merchant beacon device 120 is activated, the Wi-Fi controller 122 polls radio signals or listens for radio signals from the user computing device 110 via the antenna 121.

[0056] The example merchant point-of-sale device 130 includes an audio module 131, a payment application 133, a user interface 135, a data storage unit 136, and a communication application 139.

[0057] In the example embodiment, the audio module 131 can be any module or function of the merchant POS device 130 that captures audio input from the external environment of the merchant POS device 130. The audio module 131 can reside on the merchant POS device 130 or be logically connected to the merchant POS device 130 in any way. For example, the audio module 131 can be connected to the merchant POS device 130 via network 170. The audio module 131 is capable of audio recording. Any suitable audio recording device can be represented by the audio module 131.

[0058] In example embodiments, payment application 133 is a program, function, routine, applet, or similar entity that exists on merchant point-of-sale device 130 and performs its operations. In some example embodiments, merchant point-of-sale (“POS”) device operator 102 or other merchant system operator may install payment application 113 and / or perform feature selection on merchant point-of-sale device 130 to obtain the benefits of the technology described herein. In example embodiments, merchant POS device operator 102 may access payment application 133 on merchant POS device 130 via user interface 135. In example embodiments, payment application 133 may be associated with payment processing system 160. In another example embodiment, payment application 133 may be associated with a merchant system associated with merchant beacon device 120 and merchant camera device 140. In yet another example embodiment, there are two payment applications 133, one associated with the merchant system and the other associated with payment processing system 160.

[0059] In an example embodiment, user interface 135 enables merchant POS device operator 102 to interact with merchant POS device 130. For example, user interface 135 may be a touchscreen, a voice-based interface, or any other interface that allows merchant POS device operator 102 to provide input and receive output from applications or modules on merchant POS device 130. In an example embodiment, merchant POS device operator 102 interacts with payment application 133 via user interface 135.

[0060] In an example embodiment, data storage unit 136 includes a local or remote data storage structure suitable for storing information and accessible by the merchant POS device 130. In an example embodiment, data storage unit 136 stores encrypted information, such as HTML5 local storage.

[0061] In an example embodiment, communication application 139, such as a web browser application or a standalone application, enables the operator of merchant POS device 130 to view, download, upload, or otherwise access documents or web pages via distributed network 170. For example, communication application 139 can communicate with payment processing system 160 via network 170.

[0062] Example merchant camera device 140 includes processor 143, data storage unit 146, camera module 147, and communication application 149.

[0063] In an example embodiment, processor 143 performs one or more functions as described herein, as performed by merchant camera device 140.

[0064] In an example embodiment, data storage unit 146 includes a local or remote data storage structure suitable for storing information accessible by the merchant camera device 140. In an example embodiment, data storage unit 146 stores encrypted information, such as HTML5 local storage.

[0065] In the example embodiment, camera module 147 may be any module or function of merchant camera device 140 that captures video input or digital images of the external environment of merchant camera device 140. Camera module 147 may reside on merchant camera device 140 or be logically connected to merchant camera device 140 in any way. For example, camera module 147 may be connected to merchant camera device 140 via network 170. Camera module 147 is capable of acquiring personal images or video scans. Any other suitable image capture device may be represented by camera module 147.

[0066] In an example embodiment, communication application 149 enables merchant camera device 140 to communicate with the server of payment processing system 160.

[0067] In the example, issuer system 150 approves or rejects payment authorization requests received from payment processing system 160. In this example embodiment, issuer system 150 communicates with payment processing system 160 via network 170. In this example embodiment, issuer system 150 also communicates with an acquirer system to approve credit authorizations and make payments to payment processing system 160 and / or merchant systems. For example, the acquirer system is a third-party payment processing company.

[0068] The example payment processing system 160 includes an account management module 161, a facial recognition module 163, an audio recognition module 165, a data storage unit 166, and a transaction processing module 167.

[0069] In an example embodiment, the account management module 161 manages one or more user 101 accounts. In an example embodiment, user 101 accounts may include digital wallet accounts, email accounts, social network accounts, or any other suitable accounts associated with the payment processing system 160. In an example embodiment, the account management module 161 communicates with a payment application 113 operating on a user computing device 110 that associates user 101 with an account with the payment processing system 160. In an example embodiment, user 101 enters payment account information into their account via the payment application 113, and the account management module 161 receives the payment account information via network 170 and associates the received payment account information with user 101's account.

[0070] In an example embodiment, the facial recognition module 163 receives, via network 170 and user computing device 110, a facial image of user 101 associated with an account submitted by user 101. For example, user 101 submits a facial image when establishing an account with payment processing system 160. In the example embodiment, the facial recognition module 163 generates a facial template based on the received facial image.

[0071] In an example embodiment, the audio recognition module 165 receives, via network 170 and user computing device 110, an audio recording of user 101 associated with an account submitted by user 101. For example, user 101 submits an audio recording when establishing an account with payment processing system 160. In an example embodiment, the audio recognition module 165 generates an audio template based on the received audio recording of user 101. In an example embodiment, the audio recognition module 165 receives an audio recording of a first user 101 and compares the audio recording of the first user 101 with an audio template stored associated with a second user 101.

[0072] In an example embodiment, data storage unit 166 includes a local or remote data storage structure accessible to the payment processing system 160, suitable for storing information. In an example embodiment, data storage unit 166 stores encrypted information, such as HTML5 local storage.

[0073] In an example embodiment, transaction processing module 167 receives transaction details and a request to initiate a transaction from merchant POS device 130. Example transaction details include merchant system account information, the total transaction amount, and user 101's selection of a payment account associated with user 101's account with payment processing system 160. For example, user 101's account is a digital wallet account that includes one or more payment account information corresponding to one or more of user 101's respective payment accounts. In an example embodiment, transaction processing module 167 retrieves payment account information from user 101's account corresponding to user 101's selection of payment account in the transaction details received from merchant POS device 130. In an example embodiment, transaction processing module 167 transmits a payment authorization request to issuer system 150 or another appropriate financial institution associated with the payment account selected by user 101 for use in the transaction. Example payment authorization request may include merchant system payment account information, user 101's payment account information, and the total transaction amount. In an example embodiment, after the issuer system 150 processes the payment authorization request, the transaction processing module 167 receives approval or rejection of the payment authorization request from the issuer system 150 via network 170. In an example embodiment, the transaction processing module 167 transmits a receipt including a transaction summary to the merchant POS device 130 and / or the user computing device 110.

[0074] It will be recognized that the network connection shown is exemplary and that other methods can be used to establish a communication link between a computer and a device. Furthermore, those skilled in the art who benefit from this disclosure will recognize... Figure 1 The user computing device 110, merchant beacon device 120, merchant point-of-sale device 130, merchant camera device 140, issuer system 150, and payment processing system 160 shown may have any of several other suitable computer system configurations. For example, the user computing device 110, embodied as a mobile phone or handheld computer, may or may not include all of the components described above.

[0075] In the example embodiments, the network computing device and any other computing machine associated with the technology presented herein can be any type of computing machine, such as, but not limited to, those referenced herein. Figure 15 The machines described in more detail. Furthermore, any module associated with any of these computing machines, such as the modules described herein or any other module (scripts, web content, software, firmware, or hardware) related to the techniques presented herein, may be referenced. Figure 15 Any of the modules described in more detail herein. The computing machines described herein can communicate with each other and with other computers or communication systems on one or more networks, such as network 170. Network 170 may include any type of data or communication network, including references Figure 15Any of the aforementioned network technologies.

[0076] Example process

[0077] In the following text, the components of the example operating environment 100 are described with reference to the description. Figures 2-14 The example method shown can also be executed on other systems and in other environments. Figures 2-7 Example methods.

[0078] Figure 2 This is a block diagram illustrating a method 200 for conducting hands-free transactions using facial recognition of user 101 according to certain example embodiments.

[0079] In block 210, the merchant system registers with payment processing system 160 and installs hardware at the merchant location. See below for reference. Figure 3 The method shown describes in more detail the method for registering the merchant system with the payment processing system 160 and installing hardware at the merchant system location.

[0080] Figure 3 This is a block diagram of a method 210 for registering a merchant system with a payment processing system 160 and installing hardware at the merchant system location, according to some example embodiments. (See reference...) Figure 1 The components shown are described in method 210.

[0081] In the example embodiments described herein, the merchant system does not need to install the hardware at the example merchant system location in any particular order. Method 210 describes an example method for installing hardware at the merchant location. However, the merchant system or other systems that install merchant hardware do not need to install the merchant POS device 130, merchant camera device 140, or merchant beacon device 120 in the order described herein.

[0082] In block 310, the merchant system registers with payment processing system 160. In an example embodiment, the merchant system's agent accesses the payment processing system 160's website and registers the merchant account with payment processing system 160 via the website. In an example embodiment, the merchant system adds payment account information associated with the merchant account to the merchant account managed by payment processing system 160. In an example embodiment, the merchant system includes one or more merchant system locations. For example, the merchant system may include one or more physical store locations. An example merchant location includes one or more point-of-sale (“POS”) devices 130. In an example embodiment, one or more merchant POS device operators 102 operate one or more merchant POS devices 130 at the merchant system location.

[0083] In block 320, the merchant system operator installs payment application 133 on the merchant point-of-sale device 130. In another example embodiment, the merchant system operator uses payment application 133 pre-installed on the merchant POS device 130 to purchase the merchant POS device 130 from the payment processing system 160. In an example embodiment, the merchant POS device 130 is able to communicate with the payment processing system 160 via network 170. In an example embodiment, the merchant POS device 130 communicates with the payment processing system 160 via payment application 133. For example, the merchant POS device 130 can transmit transaction details to the payment processing system 160 via network 170 and payment application 133 to enable the payment processing system 160 to process the transaction. In another example, the merchant POS device 130 can receive a receipt from the payment processing system 160 notifying the merchant POS device operator 102 whether the transaction was successful.

[0084] In block 330, merchant beacon device 120 receives a beacon identifier code from payment processing system 160. In an example embodiment, the merchant system receives the beacon identifier from payment processing system 160 and installs or otherwise stores the beacon identifier on merchant beacon device 120. In an example embodiment, a merchant system operator installs merchant beacon device 120 within proximity to merchant POS device 130. In an example embodiment, a merchant system operator installs multiple merchant beacon devices 120, each within proximity to one or more associated merchant POS devices 130. In an example embodiment, merchant beacon device 120 is capable of broadcasting the merchant beacon identifier via a wireless medium, wherein one or more user computing devices 110 located within a threshold proximity of merchant beacon device 120 are capable of receiving the merchant beacon identifier via a wireless medium. In another example embodiment, the merchant beacon device 120 is capable of establishing a local network 170 connection with one or more user computing devices 110 located within a threshold proximity of the merchant beacon device 120, and the merchant beacon device 120 transmits the merchant beacon identifier to the one or more user computing devices 110 via the established local network 170 connection. For example, the threshold proximity depends on the network 170 communication protocol adopted by the merchant beacon device 120.

[0085] At block 340, the merchant beacon device 120, at the location of the merchant system, broadcasts a beacon identifier code via wireless communication. For example, the merchant beacon device 120 broadcasts, transmits, or otherwise transmits data including the beacon identifier to one or more user computing devices 110 located within a threshold proximity of the merchant beacon device 120 via Wi-Fi, Bluetooth, Bluetooth Low Energy (“BLE”), Near Field Communication (“NFC”), or other suitable communication protocols. In some example embodiments, prior to transmitting the merchant beacon identifier, the merchant beacon device 120 establishes a network 170 connection between the merchant beacon device 120 and one or more user computing devices 110 located within a threshold proximity of the merchant beacon device 120.

[0086] In block 350, the merchant system operator installs a merchant camera device 140 at the merchant system location to correspond with the merchant beacon device 120. In an example embodiment, both the merchant camera device 140 and the merchant beacon device 120 are installed within proximity of the merchant POS device 130. In another example embodiment, the merchant camera device 140 and the merchant beacon device 120 are installed within proximity of one or more specific merchant POS devices 130. In an example embodiment, the merchant camera device 140 is oriented to be able to capture video and / or images of the face of a user 101 standing in front of one or more merchant POS devices 130 during the checkout process. In an example embodiment, the merchant system installs the merchant camera device 140 oriented to capture video and / or images of the face of a user standing in front of a specific merchant POS device 130. In another example embodiment, the merchant system installs a merchant camera device 140 that is oriented to capture video and / or images of the faces of one or more users 101 standing within the field of view of a particular plurality of merchant POS devices 130.

[0087] At block 360, payment processing system 160 receives the merchant camera device 140 identifier and associates it with the corresponding beacon identifier code of merchant beacon device 120. In an example embodiment, the merchant system and / or payment processing system 160 configures the merchant camera device 140 such that it can communicate with payment processing system 160 via network 170. The example camera device 140 identifier includes a hardware identifier, a MAC address, or other useful or relevant identifier associated with the merchant camera device 140. In an example embodiment, payment processing system 160 includes a database containing the merchant camera device 140 identifier and associated beacon identifiers for the merchant beacon device 120 identifier at a specific merchant system location. In an example embodiment, in addition to the merchant camera device 140 identifier, the merchant camera device also transmits the merchant beacon device 120 identifier to payment processing system 160. In one example embodiment, during the setup and installation process, merchant camera device 140 can receive a merchant beacon device 120 identifier from merchant beacon device 120 via a suitable wireless communication channel. In another example embodiment, during the setup and installation process, merchant camera device 140 can establish a network 170 connection with merchant beacon device 120 and receive the merchant beacon device 120 identifier via network 170. In yet another example embodiment, payment processing system 160 receives the merchant camera device 140 identifier, retrieves one or more merchant beacon device 120 identifiers from a database, and associates the merchant camera device 140 identifier with one or more of the retrieved merchant beacon device 120 identifiers. In yet another example embodiment, after installing one or more merchant camera devices 140, a merchant system operator installs one or more merchant beacon devices 120. In this example embodiment, payment processing system 160 generates a merchant beacon device identifier associated with the merchant camera device 140 identifier and transmits the generated merchant beacon device identifier to the merchant system. In this example embodiment, the merchant system operator manually configures the merchant beacon device 120 to broadcast, send, or otherwise transmit the merchant beacon device identifier assigned by the payment processing system 160 over the network 170.

[0088] In some example embodiments, one or both of the merchant camera device 140 and the merchant beacon device 120 are components of the merchant POS device 130 or are wirelessly or physically connected to the merchant POS device 130 and controlled by one or more processors of the merchant POS device 130. In some example embodiments, certain functions described herein, such as those performed by the merchant camera device 140 and / or the merchant beacon device 120, may also be performed by the merchant POS device 130.

[0089] From block 360, method 210 proceeds to... Figure 2 Block 220.

[0090] Return to Figure 2 In block 220, user 101 registers with payment processing system 160. See below for reference. Figure 4 Method 220 describes in detail the method by which user 101 registers an account with payment processing system 160.

[0091] Figure 4 This is a block diagram illustrating a method 220 for user 101 to register an account with payment processing system 160 according to certain example embodiments. (See reference) Figure 1 The component shown is described in method 220.

[0092] In block 410, user 101 accesses payment processing system website 169. For example, user 101 accesses payment processing system 160 via web browser 118 on user computing device 110. In another example, user 101 may contact payment processing system 160 in other ways to register user 101 account.

[0093] In block 420, user 101 registers with payment processing system 160. User 101 can obtain a user account, receive appropriate applications and software to be installed on user computing device 110, request authorization to participate in hands-free payment processing, or perform any actions required by payment processing system 160. User 101 can utilize the functions of user computing device 110, such as user interface 115 and web browser 118, to register and configure user 101 account. In an example embodiment, user 101 can enter payment account information associated with one or more user 101 accounts, such as one or more credit accounts, one or more bank accounts, one or more stored value accounts, and / or other suitable accounts, into user 101 account maintained by payment processing system 160.

[0094] In block 430, user 101 downloads payment application 113 to user computing device 110. In an example embodiment, payment application 113 operating on user computing device 110 can communicate with payment processing system 160 via network 170. In an example embodiment, user 101 can configure user 101 account settings or add, delete, or edit payment account information via payment application 113. In an example embodiment, user 101 can choose to enable or disable the option for payment processing system 160 to process hands-free transactions. For example, hands-free transactions include situations where user 101 does not need to interact with user computing device 110 or requires minimal user 101 interaction with user computing device 110 to initiate a transaction with the merchant system.

[0095] In block 440, payment processing system 160 creates a facial template associated with user 101's account. See below for reference. Figure 5Method 440 describes in detail a method for creating a facial template associated with user 101 account.

[0096] Figure 5 This is a block diagram illustrating a method 440 for creating a facial template associated with user 101 account, according to some example embodiments. (See reference) Figure 1 The component shown is described in method 440.

[0097] In block 510, payment application 113 displays a request to user 101 to capture a facial image via user computing device 110. In an example embodiment, payment application 113 displays the request via user interface 115. In this example, user interface 115 may display a request that reads, “To enable hands-free transactions, we need an image of your face. Would you like to submit a facial image now?” In this example, user 101 can choose the option of taking a current photo or otherwise select a photo stored on user computing device 110.

[0098] In block 520, user 101 selects the option to capture a facial image. For example, user 101 is prompted to read "yes, I would like to take a picture now" on the user interface 115.

[0099] At block 530, payment application 113 activates camera module 117 on user computing device 110, and user 101 captures an image of their face. In an example embodiment, user interface 115 on user computing device may display a live camera feed of user 101 to help align user 101's face for capturing a facial image. In an example embodiment, payment application 113 may display a frame or other perimeter on user interface 115 on user computing device 110, within which user 101 should align their face to capture a photo of the desired size predetermined by payment processing system 160. In an example embodiment, user 101 may stimulate an object on user interface 115 to capture an image. In this example embodiment, in response to user stimulation of an object on user interface 115, camera module 117 receives a command from payment application 113 to capture an image of user 101. In another example embodiment, camera module 117 receives a command from payment application 113 to capture multiple images of user 101 as user 101 moves the camera around user 101's face. For example, each of the multiple images of user 101 may correspond to a specific pose of user 101's face. Example facial images may include digital images of user 101's face. In an example embodiment, payment processing system 160 may establish criteria for user 101 when submitting facial images. For example, payment application 113 may instruct user 101 to remove any hats, head coverings, glasses, or other objects or accessories that would obscure areas of user 101's face, allowing payment application 160 to receive a complete drawing of user 101's face.

[0100] In an example embodiment, user computing device 110 determines whether the captured facial image is a valid or invalid facial image. For example, a valid facial image conforms to criteria predetermined by payment processing system 160, while an invalid facial image does not conform to one or more criteria. For example, if user computing device 110 captures a facial image that includes an incorrect size, if part or all of user 101's face is obscured, or if the image is too dark or too bright, user computing device 110 rejects the invalid facial image and displays a request instructing user 101 to capture a subsequent facial image. In this example, user 101 captures a subsequent facial image via user computing device 110, and user computing device 110 transmits the subsequent facial image to payment processing system 160 via network 170.

[0101] At block 540, payment processing system 160 receives facial images. In another example embodiment, payment processing system 160 receives multiple facial images of user 101. For example, payment application 113 transmits one or more facial images of user 101 to payment processing system 160 via network 170. In an example embodiment, payment processing system 160 associates the received one or more facial images with user 101's account. For example, payment processing system 160 is able to identify user 101's account to associate the received one or more images because user 101 is currently logged into payment application 113 on user computing device 110 when the one or more facial images are transmitted to payment processing system 160. In some example embodiments, payment processing system 160 determines whether the received facial images are valid or invalid. For example, valid facial images meet all criteria predetermined by payment processing system 160, while invalid facial images do not meet one or more of the criteria. For example, if user 101 submits a facial image that is incorrectly sized, if part or all of user 101's face is obscured, or if the image is too dark or too bright, payment processing system 160 rejects the invalid facial image and sends a request to user computing device 110 instructing user 101 to capture a subsequent facial image for transmission to payment processing system 160. In this example, user computing device 110 receives and displays the request, user 101 captures the subsequent facial image via user computing device 110, and user computing device 110 transmits the subsequent facial image to payment processing system 160 via network 170.

[0102] In block 550, payment processing system 160 creates a facial template associated with user 101's account based on the received facial images. In another example embodiment, payment processing system 160 generates a corresponding facial template for each of a plurality of received facial images associated with user 101's account. In the example, the facial template is of a predetermined size, such as a 128-byte facial template. In the example, payment processing system 160 generates a facial template that includes a computer code representation of a digital facial image. For example, the facial template may describe key features of user 101's facial image, such as the shape, color, lines, values, space, form, texture, or other useful or relevant features of the image or a specific region of the image. In the example, the facial image is processed using a convolutional neural network to generate the facial template. In the example embodiment, payment processing system 160 stores the facial templates generated associated with user 101 in a data storage unit 166 associated with payment processing system 160. For example, the payment processing system 160 database may include tables or other means by which each user 101 account identifier is associated with a relevant facial template for user 101.

[0103] In another example embodiment, after user computing device 110 captures one or more facial images of user 101, user computing device 110 generates one or more facial templates corresponding to one or more of the one or more captured facial images of user 101. In this example embodiment, user computing device 110 transmits one or more generated facial templates to payment processing system 160 via network 170.

[0104] In block 560, payment processing system 160 deletes the received facial image. For example, payment processing system 160 uses only a facial template represented by computer code, including the facial image of user 101. In another example, payment processing system 160 saves the received facial image for future processing. For example, payment processing system 160 updates the facial template generation algorithm at a later time and generates an updated facial template corresponding to the saved facial image.

[0105] From block 560, method 440 proceeds to... Figure 4 Block 450.

[0106] Return to block 450, in Figure 4 In this process, payment processing system 160 creates an audio template associated with user 101's account. See below for reference. Figure 6 Method 450 describes a method for creating an audio template associated with user account 101.

[0107] Figure 6 This is a block diagram illustrating a method 450 for creating an audio template associated with user account 101, according to some example embodiments. (See reference) Figure 1 The component shown is described in method 450.

[0108] In block 610, payment application 113 displays a request to capture an audio recording of user 101's voice via user computing device 110. In an example embodiment, payment application 113 displays the request via user interface 115. In this example, user interface 115 may display a request read as "To enable hands-free transactions, we need recording of your voice. Would you like to submit an audio recording now?" In this example, user 101 can choose to submit a live audio recording or otherwise select a pre-recorded audio recording of user 101 stored on user computing device 110.

[0109] In block 620, user 101 selects the option to capture an audio recording. For example, user 101 is prompted to read "yes, I would like to submit an audio recording now" on the user interface 115 object.

[0110] At block 630, payment application 113 activates the audio module (not shown) on user computing device 110, and user 101 captures audio of user 101's voice. In an example embodiment, user interface 115 on user computing device may display guidelines received from payment processing system 160 to assist user 101 in submitting audio recordings. For example, payment application 113 may display an instruction to user 101 to record user 101's voice saying user 101's name. In another example, payment application 113 instructs user 101 to capture audio recordings to determine a voice password that can be used in user 101's hands-free transactions. In an example embodiment, user 101 may activate an object on user interface 115 to capture or record live audio recordings. In this example embodiment, in response to user activation of an object on user interface 115, the audio module receives a command from payment application 113 to capture user 101's audio recordings. In an example embodiment, payment processing system 160 may establish guidelines for user 101 to submit audio recordings. For example, payment application 113 may instruct user 101 to submit audio recordings no longer than a predetermined maximum duration. For example, the audio recording must not exceed 5 seconds. In another example, payment application 113 may instruct user 101 to submit an audio recording for a predetermined duration, such as 5 seconds. In an example embodiment, after user 101 initiates audio recording through an object stimulating user interface 115, payment application 113 sends a command to the audio module to stop recording user 101's voice after the predetermined duration or a predetermined maximum duration has elapsed.

[0111] In block 640, payment processing system 160 receives an audio recording of user 101's voice. For example, payment application 113 or an audio module transmits the audio recording to payment processing system 160 via network 170. In another example, payment application 113 retrieves an audio recording selected by user 101 and stored in data storage unit 116, and transmits the retrieved audio recording to payment processing system 160 via network 170.

[0112] In block 650, payment processing system 160 creates an audio template associated with the voice of user 101 based on the received user's voice. In this example, the audio template is of a predetermined size. In this example, payment processing system 160 generates a computer code representation of the audio recording of user 101. For example, the audio template may describe key features of the audio recording of user 101, such as the tone of voice of user 101 or other features of the voice of user 101. In this example embodiment, payment processing system 160 stores the generated audio template associated with user 101 in a data storage unit 166 associated with payment processing system 160. For example, the database of payment processing system 160 may include tables or other means by which each user 101 account identifier is associated with an associated audio template of user 101.

[0113] At block 660, payment processing system 160 removes the audio of the received voice recording of user 101. In an example embodiment, payment processing system 160 removes the received audio recording of user 101 to protect user 101's privacy. For example, payment processing system 160 uses only an audio template that includes a computer code representation of user 101's audio recording.

[0114] Proceed from block 660, method 440 to... Figure 2 Block 230 in the middle.

[0115] Returning to block 230, in Figure 2 In one example, user 101 enters the merchant system location and logs into the payment application 113 on the user computing device 110. In another example embodiment, before entering the merchant system location, user 101 logs into the payment application 113 and enters the merchant location with the user computing device 110 logged into the payment application 113.

[0116] In block 240, the user equipment receives the merchant beacon device 120 identifier. See below for reference. Figure 7 The method 240 described herein describes in more detail a method by which user computing device 110 receives a merchant beacon identifier broadcast by merchant beacon device 120.

[0117] Figure 7 This is a block diagram illustrating a method 240 for a user computing device 110 to receive a merchant beacon identifier broadcast by a merchant beacon device 120, according to some example embodiments. Reference Figure 1 The component shown is described in method 240.

[0118] In block 710, user 101 logs into payment application 113 on user computing device 110. In an example embodiment, user 101 has a username and password associated with a user 101 account maintained by payment processing system 160. In an example embodiment, user 101 opens payment application 113 on user computing device 110 and logs into payment application 113 by entering a username and / or password via user interface 115. In an example embodiment, when user 101 is logged into payment application 113, payment application is able to communicate with payment processing system 160 via network 170. In this example embodiment, when user 101 is not logged into payment application 113, payment application does not communicate with payment processing system 160 even if network 170 connectivity is available. In an example embodiment, user 101 can log out of payment application 113 at any time by activating one or more objects on user interface 115 of user computing device 110. In an example embodiment, after logging into payment application 113, user 101 configures one or more user 101 account settings, adds, edits, or deletes user 101 payment account information, and / or changes user 101 preferences. In some example embodiments, user 101 may be required to perform feature selection to obtain the benefits of the techniques described herein. For example, user 101 must enable one or more user 101 account settings to enable hands-free transactions according to the methods described herein.

[0119] In an example embodiment, payment application 113 may provide user 101 with options, data, configurable alerts, or other suitable features. For example, payment application 113 may include a list of merchant systems and merchant locations added to the hands-free payment transaction list according to one or more methods described herein. The list may be updated periodically from payment processing system 160. Payment application 113 may notify user 101 when they are near a configured location of added publications. Payment application 113 may provide user 101 with the option to update payment preferences. Payment application 113 may provide user 101 with a list of recent transactions. Payment application 113 may provide user 101 with any other suitable information.

[0120] In block 720, user 101 carries user computing device 110 within a threshold distance of merchant beacon device 120 at the merchant system location. In an example embodiment, user 101 enters the merchant system location. User 101 may carry user computing device 110 into the merchant location in a pocket or bag, in their hand, or in any suitable manner. The merchant system location may be a store location, a kiosk location, or any suitable physical location of the merchant system. In another example embodiment, merchant POS device operator 102 may be mobile and reach the user 101's location. For example, the merchant system may be a restaurant, and merchant POS device operator 102 may be a courier with portable merchant POS device 130.

[0121] In some example embodiments, payment application 113 may alert user 101 when user 101 is near a merchant system that accepts hands-free payments. The alert may be provided via a message on user computing device 110, via email or text, or in any suitable manner. In example embodiments, the alert may be based on user 101's location, such as that determined by a GPS module (not shown) residing on user computing device 110. For example, payment application 113 accesses GPS data from the GPS module and compares the GPS location with a list of locations of merchant systems that accept hands-free payments. For example, payment application 113 may include or access a list maintained by payment processing system 160 at the location of a merchant system that accepts hands-free payments. If a match is found by the comparison, an alert is generated and provided to user 101. This match occurs if user 101 is within a configured distance of a qualified merchant system location. In example embodiments, the alert may be configured to be issued in any suitable manner. In the example, in a business-intensive environment, the alert may be combined or presented individually. In another example, the alert may be configured to only alert user 101 the number of times configured. For example, the warning may be presented three times, but not on the fourth instance. The warning may be presented as an audible alert, vibration, a pop-up warning on the user interface 115 of the user computing device 110, or other appropriate notification.

[0122] At block 730, user computing device 110 receives a merchant beacon identifier broadcast by merchant beacon device 120. User computing device 110 identifies merchant beacon device 120 via wireless communication at the location of the merchant system. User computing device 110 can be configured to search for beacons or other wireless signals. In an example embodiment, user computing device 110 and merchant beacon device 120 establish a Wi-Fi wireless network 170 connection. In other example embodiments, user computing device 110 and merchant beacon device 120 establish a Bluetooth, BLE, NFC, or other suitable network 120 connection. Upon entering the signal range of merchant beacon device 120, user computing device 110 receives the merchant beacon identifier.

[0123] At block 740, user computing device 110 transmits the received merchant beacon identifier and user 101 account identifier to payment processing system 160. In an example embodiment, user computing device 110 transmits the data received in the merchant beacon identifier, along with the user 101 account identifier, to payment processing system 160 via network 170.

[0124] At block 750, payment processing system 160 receives a merchant beacon identifier and a user 101 account identifier. For example, payment processing system 160 receives the merchant beacon identifier and user 101 account identifier via network 170. User computing device 110 can compare the data from the merchant beacon identifier with a database of merchant beacon identifier data and merchant camera device identifier data to determine the identity of the merchant system and merchant camera device 140 associated with the merchant beacon identifier and / or verify the authenticity of the beacon.

[0125] Proceed from block 750, method 240 to... Figure 2 Block 250 in the middle.

[0126] refer to Figure 2 In block 250, merchant camera device 140 receives facial templates for each user 101 within range of merchant beacon device 120. (See reference) Figure 8 The method 250 describes a method for a merchant camera device 140 to receive a facial template for each user 101 within the range of a merchant beacon device 120.

[0127] Figure 8 This is a block diagram illustrating a method 250 for a merchant camera device 140 to receive a facial template for each user 101 within the range of a merchant beacon device 120, according to some example embodiments. Reference Figure 1 The components shown are described in method 250.

[0128] At block 810, payment processing system 160 retrieves the facial template associated with user 101's account identifier. In an example embodiment, payment processing system 160 accesses a database comprising stored facial templates of multiple users 101 having a corresponding user 101 account identifier for each user 101. For example, this database is stored in data storage unit 166.

[0129] In block 820, payment processing system 160 identifies a merchant camera device 120 associated with a merchant beacon device 120 identifier. In an example embodiment, payment processing system 160 identifies a merchant beacon identifier associated with a specific merchant camera device 140 at a payment processing system 160 and a merchant system location. In an example embodiment, payment processing system 160 identifies a merchant beacon identifier associated with multiple merchant camera devices 140 installed at a specific merchant location.

[0130] At block 830, payment processing system 160 transmits the facial template of the identified user 101 to a merchant camera device 140 associated with a merchant beacon device 120 identifier. In another example embodiment, payment processing system 160 transmits the facial template of the identified user 101 to multiple merchant camera devices 140 associated with a merchant beacon device 120 identifier. In some example embodiments, payment processing system 160 receives multiple transmissions in real time from user computing devices 101 corresponding to multiple users 101 at merchant system locations, each transmission including a user 101 account identifier and a retransmitted merchant beacon identifier. In these example embodiments, in response to receiving each such transmission, payment processing system 160 retrieves the facial template associated with the received user 101 account identifier and transmits the facial template to one or more merchant camera devices 140 at the merchant location associated with the merchant beacon identifier.

[0131] At block 840, merchant camera device 140 receives a facial template of user 101. In another example embodiment, multiple merchant camera devices 140 receive facial templates of user 101. In yet another example embodiment, according to the previously described method, merchant camera device 140 and / or multiple merchant camera devices 140 receive from payment processing system 160 one or more additional facial templates corresponding to one or more users 101 other than the immediate user 101 having user computing device 110, in a network connection with merchant beacon devices 170. For example, one or more additional facial templates are received in real time from payment processing system 160 when additional users 101 other than the immediate user 101 receive a merchant beacon device 120 identifier via wireless communication network 170 or otherwise establish a network 170 connection between their user computing device 110 and one or more merchant beacon devices 120. For example, one or more merchant camera devices 140 may receive one or more additional facial templates corresponding to one or more additional users 101 before, at the same time, or after the merchant camera device 140 receives the facial template of the real user 101.

[0132] At block 850, merchant camera device 140 adds a facial template of user 101 to the current consumer log. In an example embodiment, the current consumer log is accessible by both merchant camera device 140 and payment processing system 160. In another example embodiment, the current consumer log may additionally be accessible by merchant POS device 130 and / or available computing devices of the merchant system. In an example embodiment, payment processing system 160 includes and maintains the current consumer log, which is accessed by merchant camera device 140 via network 170. In another example embodiment, merchant camera device 140 includes and maintains the current consumer log, which payment processing system 160 can access in communication with user computing device 110 via network 170.

[0133] In block 860, merchant camera device 140 periodically updates the current consumer log. See below for reference. Figure 9 The method 860 describes in detail how a merchant camera device 140 receives a notification from a payment processing system 160 when a user 101 enters or leaves the network range of a merchant beacon device 120.

[0134] Figure 9 This is a block diagram illustrating a method 860 for a merchant camera device 140 to receive a notification from a payment processing system 160 when a user 101 enters or leaves the network range of a merchant beacon device 120, according to certain example embodiments. (See reference...) Figure 1 The components shown are described in method 860.

[0135] In block 910, when user 101, logged into a payment account, enters or leaves the network range of merchant beacon device 120, payment processing system 160 notifies merchant camera device 140. For example, as previously described, when user 101 carrying user computing device 110 enters a threshold distance from merchant beacon device 120, either merchant beacon device 120 or user 101's user computing device 110 can detect other devices and establish a wireless network 170 connection between the two devices at the merchant system location. In this example, merchant beacon device 120 transmits the merchant beacon identifier corresponding to merchant beacon device 120 to user computing device 110 via wireless network 170. For example, merchant beacon device 120 transmits the merchant beacon identifier to user computing device 110 via Wi-Fi, Bluetooth, BLE, or NFC wireless communication network 170. In this example, the user computing device 110 retransmits the received merchant beacon identifier along with the user 101 account identifier that identifies user 101 to the payment processing system 160.

[0136] In block 920, merchant camera device 140 determines whether new user 101 is within range of merchant beacon device 120. For example, if payment processing system 160 receives a new user 101 account identifier in addition to the same merchant beacon identifier, payment processing system 160 can determine that new user 101 is within range of merchant beacon device 120. In this example, payment processing system 160 can infer that new user 101 has entered the merchant location based on receiving the new user 101 account identifier. In another example, if payment processing system 160 does not receive any new user 101 account identifier and the same merchant beacon identifier within a threshold duration, payment processing system 160 can determine that no new user 101 has entered the network 170 range of merchant beacon device 120.

[0137] If the new user 101 is within range of the merchant beacon device 120, method 860 proceeds to... Figure 8 Block 830 in the example. For instance, in addition to the same merchant beacon identifier, payment processing system 160 also receives a new user 101 account identifier. In this example, payment processing system 160 infers that the new user 101 has entered the merchant location based on the received new user 101 account identifier and the same merchant beacon identifier previously received from the first user 101.

[0138] Return to Figure 8 In block 810, payment processing system 160 retrieves a facial template associated with the new user 101's account identifier. In an example embodiment, payment processing system 160 transmits the facial template to one or more suitable merchant camera devices 140 and, according to... Figure 8In the example method described above in block 250, one or more merchant camera devices 140 add the facial template of the new user 101 to the current consumer log.

[0139] Return to Figure 9 In block 920, if no new user 101 is found within the range of merchant beacon device 120, method 860 proceeds to block 930. For example, payment processing system 160 does not accept any new user 101 account identifier and the same merchant beacon identifier within a threshold duration, and determines that no new user 101 has entered the network 170 range of merchant beacon device 120.

[0140] At block 930, merchant camera device 140 determines whether user 101 has moved out of range of merchant beacon device 120. In an example embodiment, user computing device 110 continues to receive merchant beacon identifiers from merchant beacon device 120 and transmits these identifiers along with user 101's account identifier to payment processing system 160. In this example embodiment, user computing device 110 may periodically transmit information including the merchant beacon identifier and user 101's account identifier to payment processing system 160 as long as user computing device 110 continues to detect merchant beacon device 120 and receives the merchant beacon device 120 identifier via periodic scanning. For example, user computing device 110 scans merchant beacon device 120 every 5 seconds. In another example embodiment, user computing device 110 may periodically transmit information including the merchant beacon identifier and user 101's account identifier to payment processing system 160 as long as user computing device 110 maintains a wireless network connection 170 with merchant beacon device 120. For example, user computing device 110 can transmit this information to payment processing system 160 every 5 seconds. In this example embodiment, if payment processing system 160 stops receiving information from user computing device within a predetermined number of intervals, payment processing system 160 can determine that the corresponding user 101 has moved out of the range of merchant beacon device. In this example embodiment, if payment processing system 160 continues to receive information transmitted by user computing device 110 at expected intervals, payment processing system 160 determines that user 101 is still within the network 170 range of merchant beacon device 120.

[0141] If no current user 101 is removed from the range of merchant beacon device 120, method 860 proceeds to... Figure 2 Block 260. For example, payment processing system 160 continues to receive, at expected intervals, the merchant beacon identifier and user 101 account identifier received from user computing device 110 and determines that user 101 is still within the network 170 range of merchant beacon device 120.

[0142] Back to block 260, in Figure 2In the middle, user 101 initiated a transaction at merchant POS device 130.

[0143] Back Figure 9 In block 930, if the current user 101 moves out of the range of merchant beacon device 120, method 860 proceeds to block 940.

[0144] At block 940, merchant camera device 140 receives a notification from payment processing system 160 that current user 101 is outside the network range of merchant beacon device 120. In another example embodiment, merchant camera device 140 receives a notification from payment processing system 160 indicating that user computing device 110 associated with current user 101 has stopped sending notifications including the identifier of merchant beacon device 120 to payment processing system 160. For example, merchant camera device 140 receives the user 101 account identifier associated with current user 101, who is associated with user computing device 110 that is outside the network range or has stopped sending notifications including the identifier of merchant beacon device 120 to payment processing system 160, accesses the current consumer log, and identifies the entry corresponding to current user 101. For example, the current consumer log is maintained by payment processing system 160, and merchant camera device 140 accesses the current consumer log via network 170 by communicating with payment processing system 160. In another example embodiment, the payment processing system 160 does not send a notification that the current user 101 is outside network range to the merchant camera device 140. In this example embodiment, the payment processing system 160 accesses the current consumer log and deletes the current user 101's facial template from the current consumer log.

[0145] At block 950, merchant camera device 140 deletes the facial template of current user 101 from the current consumer log. For example, the current consumer log includes a table, and merchant camera device 140 deletes or requests deletion of entries or rows corresponding to data associated with current user 101, receiving a notification for current user 101. In another example embodiment, payment processing system 160 accesses the current consumer log and deletes the facial template of current user 101 from it.

[0146] From block 950, method 860 proceeds to... Figure 2 Block 260 in the middle.

[0147] Return to Figure 2 In block 260, user 101 initiates a transaction at merchant point-of-sale device 130. See below for reference. Figure 10The method 260 described herein provides a more detailed method for initiating a transaction at a merchant point-of-sale device 130 by user 101. In the example embodiment described herein, user 101 initiates a “hands-free transaction” at merchant POS device 130. The example hands-free transaction does not require any interaction with user computing device 110 on a portion of user 101. In another example, the hands-free transaction requires only minimal interaction with user computing device 110 by user 101.

[0148] Figure 10 This is a block diagram of a method 260 for initiating a transaction at a merchant's point-of-sale device 130 by user 101, according to some example embodiments. (See reference) Figure 1 The components shown are described in method 260.

[0149] At block 1010, user 101 approaches merchant point-of-sale (POS) device 130. In an example embodiment, prior to approaching POS device 130, user 101 browses the merchant system location and selects one or more items to purchase. In this example embodiment, user 101 may collect one or more items and transport them to POS device 130 via a physical basket or shopping cart, or otherwise.

[0150] In block 1020, merchant POS device operator 102 totals the items purchased by user 101. In an example embodiment, merchant POS device operator 102 scans barcodes affixed to one or more items or otherwise enters descriptions and prices associated with one or more items into merchant POS device 130. In an example embodiment, after scanning or manually entering an item into merchant POS device 130, merchant POS device operator 102 activates an object on user interface 135 of merchant POS device 130 to command merchant POS device 130 to total the item. In an example embodiment, merchant POS device 130 displays the total to user 101 via user interface 135.

[0151] At block 1030, the merchant point-of-sale (POS) device 130 operator prompts the user 101 to select a payment option. In an example embodiment, the merchant POS device 130 displays one or more payment options that the user 101 can choose to use in a transaction. Example payment options may include payments via a payment application 113 associated with the payment processing system 160, cash payments, check payments, credit card payments, debit card payments, and / or any other payment method that the merchant system can or wishes to accept from the user 101. In an example embodiment, one or more payment options are displayed as objects on the user interface 135 and, in response to the user 101 instructing the merchant POS device operator 102 to make a selection, which can be made by the merchant POS device operator 102.

[0152] In another example embodiment, the payment processing system 160 or the merchant camera device 140 sends a notification to the merchant point-of-sale device 130 that user 101 can initiate a transaction via payment application 113. In this example embodiment, the merchant point-of-sale device 130 notifies its operator that user 101 can participate in a transaction using payment application 113. In this example embodiment, the merchant point-of-sale device 130 operator requests confirmation from user 101 that user 101 wishes to initiate a transaction using payment application 113.

[0153] In block 1040, user 101 instructs merchant point-of-sale (POS) device operator 102 to initiate a transaction via payment application 113. In an example embodiment, in response to receiving a verbal request from user 101 to select payment application 113 as a payment option, merchant POS device operator 102 activates an object on the user interface 135 of the merchant POS device 130 corresponding to the payment option of payment application 113.

[0154] At block 1050, merchant point-of-sale (POS) operator 102 selects an option on merchant POS device 130 to initiate a transaction using payment application 113. In an example embodiment, after merchant POS operator 102 selects to initiate a transaction using payment application 113, merchant POS device 130 displays a confirmation screen. The example confirmation screen may display information summarizing the potential transaction and consisting of one or more of the following: the total transaction amount, a description of one or more items purchased by user 101, and an instruction from user 101 to select payment application 113 as the transaction payment method. The example confirmation screen may further display options to confirm or cancel the transaction. In an example embodiment, user 101 reviews the confirmation screen, determines that the information displayed on the confirmation screen is correct, decides to continue the transaction, and instructs merchant POS operator 102 to select the option to confirm the transaction via user interface 135.

[0155] In another example embodiment, user 101 decides to abandon the transaction because the information is incorrect or because user 101 changes his mind or decides not to purchase the item. In yet another example embodiment, the confirmation screen further includes an option to edit transaction details. In this example embodiment, merchant POS device operator 102 instructs user 101 to select the option to edit transaction details, and then edits, adds, or deletes one or more items in the transaction or edits payment details or the payment method of the transaction.

[0156] At block 1060, merchant point-of-sale device 130 transmits a request to identify user 101 to payment processing system 160. In the example embodiment, the merchant point-of-sale device transmits a request to process the transaction along with a request to identify user 101.

[0157] For example, at this time, payment processing system 160 knows, based on the current consumer log maintained by the payment processing system and accessed and edited by merchant beacon device 120, that one or more 110s are physically located in the merchant system location. Additionally, payment processing system 160 knows, based on processing a transaction processing request received from merchant POS device 130, that user 101 has initiated a transaction using payment application 113. However, payment processing system 160 may not know exactly which user 101 requested processing a hands-free transaction involving payment application 113.

[0158] In block 1070, payment processing system 160 receives a request from merchant point-of-sale device 130 to identify user 101. For example, payment processing system 160 receives the request to identify the user and the request to process the transaction via network 170.

[0159] At block 1080, payment processing system 160 transmits a request to identify user 101 to merchant camera device 140. For example, payment processing system 160 transmits the request to identify the user to merchant camera device 140 via network 170. In another example embodiment, merchant camera device 140 has already transmitted the identity of user 101 to payment processing system 160 or to merchant point-of-sale device 130, and payment processing system 160 does not need to transmit a request to merchant camera device 140 to identify user 101.

[0160] Proceed from block 1080, 260 to Figure 2 Block 270.

[0161] Return to Figure 2 In block 270, merchant camera device 140 identifies user 101 via facial recognition. See below for reference. Figure 11 The method 270 describes in detail a method by which a merchant camera device 140 identifies a user 101 via facial recognition.

[0162] Figure 11 This is a block diagram illustrating a method 270 for identifying a user 101 via facial recognition by a merchant camera device 140, according to some example embodiments. (See reference) Figure 1 The components shown are described in method 270.

[0163] In block 1110, merchant camera device 140 captures video of user 101. In an example embodiment, in response to receiving a request to identify user 101, merchant camera device 140 activates camera module 147 to begin capturing video of the area surrounding merchant camera device 140. In an example embodiment, merchant POS device 130 captures a video feed of user 101's face. In another example embodiment, merchant camera device 140 continuously captures but does not record video feeds of its surroundings. In this example embodiment, when merchant camera device 140 receives a request to identify user 101 from payment processing system 160, the merchant camera device begins recording video feeds for a threshold time period. In an example embodiment, user 101 may move during the recording of video feeds by merchant camera device 140. In an example embodiment, processor 143 of merchant camera device 140 extracts a facial image by determining specific frames of the video feed and the region of the video feed instance corresponding to the user's face.

[0164] At block 1120, merchant camera device 140 extracts a facial image of user 101 from the captured video. In an example embodiment, merchant camera device 140 determines frames of the captured video to provide an image of user 101's face and extracts frames from the captured video that include the facial image of user 101.

[0165] In some other example embodiments, the merchant camera device 140 determines frames of the captured video to provide images of the faces of multiple users 101. For example, the frames include images of the faces of a first user 101, a second user 101, and a third user 101 located at different positions in the image. In this example, a merchant camera device 140 captures video of the environment corresponding to an area near multiple merchant POS devices 130. In this example embodiment, the merchant camera device 140 can determine, from the extracted images, which specific merchant POS device 130 corresponds to each of the multiple faces of the multiple users 101.

[0166] In an example embodiment, the merchant camera device 140 is mounted at an angle parallel to a plurality of adjacent merchant POS devices 130. The field of view of the merchant camera device 140 can be divided into borders, such that each border is assigned to a specific merchant POS device 130. In this example embodiment, since one or more users 101 are located between any two adjacent merchant POS devices 130, the borders may overlap. In this example embodiment, each border is assigned to at most two adjacent merchant POS devices 130. In this example embodiment, the merchant camera device 140 is mounted parallel to a plurality of adjacent merchant POS devices 130, and the size of the detected facial image of user 101 is proportional to the distance of user 101 from the merchant POS device 130 detected in the image. In this example embodiment, based on the position of the facial image relative to the border, the merchant camera device 140 assigns each detected facial image to one or two specific merchant POS devices 130.

[0167] In another example embodiment, the merchant camera device 140 is mounted at an angle not parallel to multiple adjacent merchant POS devices 130. In this example embodiment, the merchant camera device 140 is capable of monitoring an environment including more adjacent merchant POS devices 130 than in the example embodiment where the merchant camera device 140 is mounted parallel to adjacent merchant POS devices 130. In this example embodiment, however, the size of the detected facial image is not proportional to the distance between the detected merchant POS terminal in the image and the detected facial image. In this example embodiment, based on the position of the facial image relative to the border, the merchant camera device 140 assigns each detected facial image to one or two specific merchant POS devices 130.

[0168] In block 1130, merchant camera device 140 generates a facial template from the captured facial image. In this example, the facial template is of a predetermined size, for example, a 128-byte facial template. In this example, payment processing system 160 generates a facial template that includes a computer code representation of the digital facial image. For example, the facial template may describe key features of the user 101's facial image, such as the shape, color, lines, values, space, form, texture, or other useful or relevant features of the image or a specific region of the image. In another example, the facial image is processed using a convolutional neural network to generate the facial template. In this example embodiment, merchant camera device 140 stores the generated facial template in a data storage unit 146 associated with merchant camera device 140. For example, the merchant camera device 140 database includes a log of facial templates of current consumers, wherein the merchant camera device 140 stores the generated facial templates.

[0169] In some other example embodiments, as a user 101 enters and leaves the vicinity of one or more merchant POS devices 130 during a certain time period, the merchant camera device 140 continuously captures video feeds around them. In this example embodiment, the merchant camera device 140 processor 147 is capable of continuously monitoring the incoming video feeds to detect faces from the extracted frames of the video feeds. In this example embodiment, each time the camera module 147 detects the presence of one or more faces in the video feed, the merchant camera device 140 extracts frames of the video feed consisting of one or more facial images of one or more corresponding detected faces and creates a facial template based on the extracted one or more facial images. In this example embodiment, when a facial template is generated, the merchant camera device 140 stores it in a log of the current consumer's facial templates. In this example embodiment, when the merchant camera device 140 generates a subsequent facial template, the merchant camera device 140 determines whether the generated subsequent facial template is similar to or within a threshold compared to any facial template already stored in the log of the current consumer's facial templates. If the generated subsequent facial template resembles or falls within any of the facial templates already stored in the log, the merchant camera device 140, after associating the facial template with one or more specific merchant POS devices 130 based on the position of the relevant facial image in the extracted captured video frames, adds the facial template to the current consumer's facial template log. If the generated subsequent facial template does not resemble or falls within a threshold of any facial template already stored in the current consumer's facial template log, the merchant camera device 140 deletes or otherwise ignores the generated facial template and / or does not process it. In this example embodiment, if the merchant camera device 140 determines that the current facial image is no longer within the range of the video feed, it deletes the corresponding facial template from the current consumer's facial template log.

[0170] In block 1140, merchant camera device 140 detects captured video and extracted facial images. For example, merchant camera device 140 does not store the captured images or videos. In this example, the facial template generated by merchant camera device 140 includes a computer code representation of user 101's facial image. In this example, after generating the facial template, or after capturing a video or image, or after extracting an image from a video, and after a threshold time has elapsed, merchant camera device 140 deletes any captured or extracted video or images.

[0171] At block 1150, merchant camera device 140 retrieves facial templates from the current consumer log. For example, the current consumer log includes facial templates received from payment processing system 160 corresponding to all current users 101, whose associated user computing devices 110 are located within network distance of merchant beacon device 120.

[0172] In block 1160, the merchant camera device 140 compares the facial template generated from the captured facial image with the facial template from the current consumer log.

[0173] At block 1170, merchant camera device 140 determines whether there is a match between the generated facial template and one of the facial templates from the current consumer log. In this example embodiment, when merchant camera device 140 determines that there is a match between a first facial template in the current consumer's facial template log and a second facial template in the current consumer's log, merchant camera device 140 determines which merchant's one or two POS devices 130 the second facial template is associated with. In this example embodiment, merchant camera device 140 updates the entry for the second facial template by assigning the second facial template to annotations of the specific one or two merchant POS devices 130. For example, as previously described, the facial image on which the facial template is based is located within a border of a region of video or image corresponding to one or two specific merchant POS devices 130. In this example, when merchant camera device 140 generates a facial template for a facial image, merchant camera device 140 associates the generated facial template with the specific one or two merchant POS devices 130. For example, if the facial image is within the border in front of the first merchant POS device 130, the resulting facial template can be associated with the first merchant POS device 130. In another example, if the facial image is within the border between the first merchant POS device 130 and the second, adjacent merchant POS device 130, the resulting facial template can be associated with both the first merchant POS device 130 and the second merchant POS device 130.

[0174] If the facial template from the current consumer log matches the generated facial template, method 270 proceeds to block 1180. In another example embodiment, if the facial template from the current consumer log matches the facial template from the log of the current consumer's facial template, method 270 proceeds to block 1180.

[0175] At block 1180, merchant camera device 140 transmits a facial template corresponding to the identified user, from the current consumer log, to payment processing system 160. For example, the transmitted facial template includes annotations assigning the facial template to one or two specific merchant POS devices 130.

[0176] Proceed from block 1180, method 270 to... Figure 2 Block 295. In this example embodiment, the payment processing system 160 identifies the user 101 who requested the transaction and the specific merchant POS device 130 assigned to the user 101 and continues to process the transaction.

[0177] Returning to block 1170, if no facial template from the current consumer log matches the generated facial template, method 270 proceeds to... Figure 2 Block 280. In another example embodiment, if no face template from the current consumer log matches any face template from the current consumer's face template log, method 270 proceeds to... Figure 2 Block 280.

[0178] Return to Figure 2 In block 280, payment processing system 160 identifies user 101 via voice recognition. In another example embodiment, payment processing system 160 identifies user 101 without voice recognition. Instead, it proceeds to identifying user 101 via a challenge and response mechanism. For example, merchant camera device 140 and / or payment processing system 160 cannot identify the user via facial recognition, and / or if merchant camera device 140 and / or payment processing system 160 identifies two or more users 101 via facial recognition. In the following, references... Figure 12 Method 280 is described in more detail as a method for a payment processing system 160 to identify a user 101 via voice recognition.

[0179] Figure 12 This is a block diagram illustrating how a payment processing system 160 identifies user 101 via voice recognition, according to certain example embodiments. (Reference) Figure 1 The component description method shown is 280.

[0180] In some example embodiments, it is necessary to identify user 101 via voice recognition because merchant camera device 140 and / or payment processing system 160 cannot identify user 101 based on facial recognition. For example, merchant camera device 140 cannot extract enough facial images of user 101 from the video feed to generate a facial template. In another example, merchant camera device 140 cannot find a match for a facial template for user 101 in the current consumer log. In yet another example, merchant camera device 140 identifies a matching facial template of user 101 in the current consumer log; however, the identified facial template of user 101 is assigned to two adjacent merchant POS devices 130. In this example, user 101 must identify themselves via voice recognition at the POS device 130 where user 101 initiated the transaction.

[0181] At block 1210, merchant camera device 140 sends a notification that it cannot recognize user 101 to payment processing system 160. In another example embodiment, merchant camera device 140 sends a notification that it has recognized user 101, but cannot determine which of the two adjacent merchant POS devices 130 user 101 is using in the transaction.

[0182] In block 1220, payment processing system 160 retrieves the audio template corresponding to user 101 from the current consumer log. As previously described, when user 101 creates an account with payment processing system 160, user 101 may submit an audio recording of user 101's voice to payment processing system 160. In this example, payment processing system 160 creates an audio template corresponding to user 101 based on the received audio recording. In another example embodiment, user 101 does not submit an audio recording to payment processing system 160 when creating user 101's account. In this other example embodiment, payment processing system 160 does not have an audio template associated with user 101's account and cannot verify user 101 via speech recognition.

[0183] At block 1230, payment processing system 160 transmits a request to merchant point-of-sale device 130 to record user 101's audio. For example, payment processing system 160 transmits the request via network 170.

[0184] In block 1240, merchant point-of-sale device 130 displays a request to user 101 to record audio. For example, merchant POS device 130 may display an instruction to user 101 to record an audio recording. For example, user 101 may be instructed to speak the same words in the same tone as when user 101 created an audio template for payment processing system 160 when user 101's account was created.

[0185] At block 1250, merchant point-of-sale device 130 records the voice input of user 101 and transmits it to payment processing system 160. For example, merchant POS device operator 102 can activate an object on user interface 135 to activate audio module 131 to receive audio input from user 101, which is then retransmitted by merchant POS device 130 to payment processing system 160 via network 170. In an example embodiment, merchant POS device 130 transmits a merchant POS device 130 identifier and the retransmitted audio input from user 101.

[0186] In block 1260, payment processing system 160 compares the received voice input with an audio template corresponding to user 101 retrieved from the current consumer log. For example, payment processing system 160 receives the audio input of user 101 from merchant POS device 130 via network 170.

[0187] At block 1270, payment processing system 160 determines whether there is a match between the received voice input and one of the audio templates retrieved from the current consumer log. In an example embodiment, payment processing system 160 compares one or more features between each of the received voice input and the audio templates retrieved from the current consumer log. In an example embodiment, if the similarity between the received voice input and a particular audio template exceeds a predetermined threshold, payment processing system 160 determines that the received voice input matches the particular audio template. In another example embodiment, if the similarity between the received voice input and a particular audio template is less than a predetermined threshold, payment processing system 160 determines that the received voice input does not match the particular audio template.

[0188] If the audio template from the current consumer log matches the received voice input, method 270 proceeds to block 1280. For example, the similarity between a specific audio template from the current consumer log and the received audio recording of user 101 exceeds a predetermined threshold and the payment processing system 160 determines that there is a match.

[0189] In block 1280, payment processing system 160 assigns user 101 to merchant point-of-sale device 130 in the current consumer log. For example, payment processing system 160 identifies user 101 associated with an audio template that matches the received audio recording of user 101. Furthermore, payment processing system 160 identifies the merchant POS device 130 to which user 101 should be assigned based on the merchant POS device 130 identifier received from the merchant POS device 130 and the audio recording of user 101.

[0190] Proceed from block 1290, method 280 to Figure 2 Block 295. For example, payment processing system 160 successfully identifies user 101 by comparing the received recording of user 101 with an audio template from the current consumer log. In this example, payment processing system 160 can proceed to process transactions involving the identified user 101.

[0191] Returning to block 1270, if no audio template in the current consumer log matches the received voice input, method 280 proceeds to... Figure 2 Block 290. In another example embodiment, the merchant POS device 130 cannot receive the audio recording of user 101 or the received audio recording of user 101 is insufficient and cannot be used to identify user 101.

[0192] Return to Figure 2 In block 290, the merchant system POS operator 102 identifies user 101 via a challenge and response. References will be made below. Figure 3The method 290 describes in detail a method by which a merchant POS device operator 102 identifies a user 101 through challenges and responses.

[0193] Figure 13 This is a block diagram illustrating a method by which a merchant POS device operator 102 identifies a user 101 via challenges and responses, according to certain example embodiments. Reference Figure 1 The components shown are described in method 290.

[0194] In block 1310, POS device operator 102 issues a challenge to user 101. In one example, merchant POS device operator 102 asks user 101 for their abbreviation. In another example, merchant POS device operator 102 asks user 101 for the last four digits of their phone number. In yet another example, merchant POS device operator 102 asks user 101 for their configured password. Any appropriate challenge is issued by merchant POS device operator 102. In the example embodiment, the response to the challenge does not provide any security or privacy information.

[0195] In block 1320, user 101 provides a challenge response. As described in the example challenge, the response could be an abbreviation of user 101, the last four digits of user 101's phone number, or a configured password. Any configured challenge response can be utilized. In some embodiments, the response could be a spoken response, a gesture, keystrokes, display of an ID card, or any suitable response.

[0196] In block 1330, point-of-sale (POS) operator 102 inputs a response to merchant POS device 130. Merchant POS operator 102 inputs the challenge response from user 101. In this example, if user 101 indicates that the abbreviation for user 101 is “AC”, then merchant POS operator 102 inputs “AC” into the payment application 133 of merchant POS device 130. In this example, the user interface 135 of merchant POS device 130 displays a request to input user 101's response. Merchant POS operator 102 inputs the response via a virtual or physical keyboard, voice dictation, or any suitable method. In an alternative example, user 101 inputs the response to user interface 135 of merchant POS device 130.

[0197] In block 1340, merchant point-of-sale device 130 displays potential user 101 based on the challenge response. A list of users 101 associated with the challenge response is displayed on merchant POS device 130 at merchant POS device operator 102. For example, if 10 consumers are near merchant beacon device 120, merchant POS device 130 receives a challenge response associated with each of the 10 consumers from payment processing system 160. When merchant POS device 130 receives a challenge response input, only the potential user 101 associated with the challenge response is displayed to merchant POS device operator 102.

[0198] In another embodiment, the merchant POS device 130 or the payment processing system 160 that handles the challenge presents an additional challenge until a single matching user 101 remains.

[0199] In this example, if merchant POS device operator 102 enters "AC" as an abbreviation for user 101 associated with a transaction, then only potential users 101 with those abbreviations will be displayed to merchant POS device operator 102 by payment application 133. Payment application 133 accesses a database on payment processing system 160 or another computing device and identifies the abbreviations of potential users 101 who have provided tokens. Payment application 133 identifies one or more potential users 101 with the abbreviation "AC" and displays the identified user 101 accounts to merchant POS device operator 102. In this example, two of the ten consumers near merchant beacon device 120 have the abbreviation "AC". The user 101 accounts of the two consumers are displayed to merchant POS device operator 102.

[0200] In some example embodiments, all nearby consumers with tokens transmitted to the merchant POS device 130 are presented to the merchant POS device operator 102, and the merchant POS device operator 102 selects the appropriate user 101 account.

[0201] Payment application 133 can display a photograph of a potential user 101 account presented to merchant POS device operator 102. For example, each user 101 can associate a photograph with its account. When merchant POS device 130 presents one or more potential users 101 to merchant POS device operator 102, merchant POS device operator 102 can match the photograph with the user 101 who will conduct the transaction, selecting the appropriate user 101 account. Instead of or in addition to the photograph, other identifying information can be presented. For example, the name of user 101 can be displayed, indicating that merchant POS device operator 102 can identify the potential user 101 by that name. Any other suitable identifying information can be presented.

[0202] At block 1350, the merchant point-of-sale (POS) operator 102 selects the user 101 account to be used in the transaction. After recognizing the displayed photo of user 101, the merchant POS operator 102 can input user 101's selection by activating the user interface 135 control associated with the photo, or by inputting the selection in any appropriate manner. If the photo does not match any of the potential users, the merchant POS operator 102 can cancel the transaction, notify user 101 of the discrepancy, or perform any other appropriate action.

[0203] In the example, only a single user 101 account is presented in the list of potential users 101. If only a single user 101 account is identified, the method continues as the merchant POS device operator 102 verifies that the displayed photo matches user 101. If the photo does not match, the merchant POS device operator 102 can cancel the transaction, notify user 101 of the discrepancy, or perform any other appropriate action.

[0204] From block 1350, method 290 proceeds to... Figure 2 Block 295.

[0205] Return to Figure 2 In block 295, the transaction is processed. See below for reference. Figure 14 Method 290 describes in more detail the method for processing transactions.

[0206] Figure 14 This is a block diagram illustrating a method 295 for processing transactions, according to some example embodiments. (See reference) Figure 1 The component shown is described in method 295.

[0207] At block 1410, payment processing system 160 transmits the identified user 101's account information to merchant point-of-sale device 130. For example, user 101's account in payment processing system 160 includes payment account information associated with one or more payment accounts of user 101. Example payment accounts for user 101 include credit accounts, bank accounts, stored-value accounts, gift cards, merchant system fee accounts, or other related user 101 accounts. In an example embodiment, merchant POS device 130 does not receive complete payment information for one or more of user 101's payment accounts. For example, payment information may be obfuscated or intercepted. For example, merchant POS device 130 may only receive abbreviated payment information sufficient for user 101 to identify the payment account via user interface 135.

[0208] At block 1420, merchant point-of-sale device 130 displays the account of the identified user 101. In an example embodiment, merchant POS device 130 does not display complete account information for user 101 on user interface 135. For example, for a credit card account, merchant POS device user interface 135 may only display the name associated with the credit card, the last four digits of the credit card number, and the expiration date.

[0209] At block 1430, merchant point-of-sale (POS) operator 102 selects user 101's account for a transaction and confirms the transaction using user 101's permission. For example, user 101 may view account options displayed directly on user interface 135 or listen to merchant POS operator 102 reading user 101's payment account options. In an example embodiment, user 101 selects a payment account option for use in a transaction. For example, user 101 decides to use a credit card account. In an example embodiment, user 101 communicates the selection of the payment account to be used in the transaction to merchant POS operator 102. In an example embodiment, in response to user 101's instruction to receive the payment account, merchant POS operator 102 selects the payment account indicated by user 101 on merchant POS user interface 135.

[0210] At block 1440, merchant POS device 130 transmits transaction details to payment processing system 160. For example, merchant POS device 130 transmits transaction details including the total transaction amount, the merchant system's account identifier, and the selected payment option for user 101's account. In an example embodiment, payment processing system 160 receives the transaction details via network 170. In an example embodiment, payment processing system 160 retrieves payment account information from the selected user 101 account corresponding to the received payment account selection for user 101.

[0211] At block 1450, payment processing system 160 transmits a transaction authorization request to issuer system 150. In an example embodiment, payment processing system 160 generates a transaction authorization request including user 101 payment account information, merchant system payment account information, and the total transaction amount. In an example embodiment, payment processing system 160 receives a transaction authorization request from payment processing system 160 via network 170.

[0212] At block 1460, issuer system 150 approves the transaction authorization request and transmits the transaction authorization permission to payment processing system 160. In an example embodiment, issuer system 150 subtracts user 101's current account balance from the credit limit associated with user 101's credit account. In an example embodiment, issuer system 150 authorizes the transaction authorization if the difference between the credit limit and the credit balance is a positive number greater than the total amount of the transaction. For example, if the transaction is approved, user 101 will not exceed his credit limit, therefore, issuer system 150 authorizes the transaction authorization request. In an example embodiment, payment processing system 160 receives the transaction authorization permission from issuer system 150. In another example embodiment, in response to issuer system 150 rejecting the transaction authorization request, payment processing system 160 receives the rejection of the transaction authorization request. In an example embodiment, if issuer system 150 authorizes the transaction authorization request, issuer system 150 credits the merchant system account and adds the fee to user 101's credit statement, bank account statement, or other appropriate invoice.

[0213] At block 1470, payment processing system 160 transmits a transaction receipt to merchant point-of-sale device 130. For example, payment processing system 160 transmits notification of transaction approval (confirmation) or rejection (rejection) to merchant POS device 130 via network 170. In an example embodiment, merchant POS device 130 prints a receipt summarizing a successful or unsuccessful transaction via a printer device that is a component of or otherwise communicatively coupled to merchant POS device 130. In another example embodiment, payment processing system 160 may transmit the transaction receipt to user computing device 110 for display on the user computing device. In yet another example embodiment, merchant POS device 130 transmits the transaction receipt to user computing device 110 for display on the user computing device.

[0214] Other example embodiments

[0215] Figure 15 A computing machine 2000 and module 2050 are depicted according to certain example embodiments. The computing machine 2000 may correspond to any of the various computers, servers, mobile devices, embedded systems, or computing systems presented herein. Module 2050 may include one or more hardware or software elements configured to enable the computing machine 2000 to perform the various methods and processing functions provided herein. The computing machine 2000 may include various internal or attached components such as a processor 2010, a system bus 2020, a system memory 2030, a storage medium 2040, an input / output interface 2060, and a network interface 2070 for communicating with a network 2080.

[0216] The computer machine 2000 can be implemented as a conventional computer system, an embedded controller, a laptop computer, a server, a mobile device, a smartphone, a set-top box, an information station, an in-vehicle information system, one or more processors associated with a television set, a custom machine, any other hardware platform, or any combination or combination thereof. The computer machine 2000 can be a distributed system configured to operate using multiple computers interconnected via a data network or bus system.

[0217] Processor 2010 may be configured to execute code or instructions to implement the operations and functionality described herein, manage request flows and address mappings, and perform computations and generate commands. Processor 2010 may be configured to monitor the operation of components in computer machine 2000. Processor 2010 may be a general-purpose processor, processor core, multiprocessor, reconfigurable processor, microcontroller, digital signal processor (“DSP”), application-specific integrated circuit (“ASIC”), graphics processing unit (“GPU”), field-programmable gate array (“FPGA”), programmable logic device (“PLD”), controller, state machine, gated logic, discrete hardware component, any other processing unit, or any combination or multiplex thereof. Processor 2010 may be a single processing unit, multiple processing units, a single processing core, multiple processing cores, dedicated processing core, coprocessor, or any combination thereof. According to some embodiments, processor 2010, together with other components of computer machine 2000, may be a virtualized computer executing within one or more other computers.

[0218] System memory 2030 may include non-volatile memory, such as read-only memory (“ROM”), programmable read-only memory (“PROM”), erasable programmable read-only memory (“EPROM”), flash memory, or any other device capable of storing program instructions or data with or without applied power. System memory 2030 may also include volatile memory, such as random access memory (“RAM”), static random access memory (“SRAM”), dynamic random access memory (“DRAM”), and synchronous dynamic random access memory (“SDRAM”). Other types of RAM may also be used to implement system memory 2030. System memory 2030 may be implemented using a single memory module or multiple memory modules. Although system memory 2030 is depicted as part of computer machine 2000, those skilled in the art will recognize that system memory 2030 may be separated from computer machine 2000 without departing from the scope of the invention. It should also be understood that the system memory 2030 may include or operate in conjunction with a non-volatile storage device (such as storage medium 2040).

[0219] Storage medium 2040 may include hard disks, floppy disks, optical disc read-only memory (“CD-ROM”), digital versatile optical discs (“DVD”), Blu-ray discs, magnetic tape, flash memory, other non-volatile storage devices, solid-state drives (“SSD”), any magnetic storage device, any optical storage device, any electrical storage device, any semiconductor storage device, any physical storage device, any other data storage device, or any combination or combination thereof. Storage medium 2040 may store one or more operating systems, applications, and program modules, such as module 2050, data, or any other information. Storage medium 2040 may be part of or connected to computer machine 2000. Storage medium 2040 may also be part of one or more other computer machines communicating with computer machine 2000, such as servers, database servers, cloud storage, network-attached storage, etc.

[0220] Module 2050 may include one or more hardware or software elements configured to enable computer machine 2000 to perform the various methods and processing functions provided herein. Module 2050 may include one or more sequences of instructions stored in software or firmware form in conjunction with system memory 2030, storage medium 2040, or both. Thus, storage medium 2040 may represent an instance of a machine or computer-readable medium on which instructions or code may be stored for execution by processor 2010. Machine or computer-readable medium may generally refer to any medium used to provide instructions to processor 2010. Such machine or computer-readable media associated with module 2050 may include computer software products. It should be understood that computer software products including module 2050 may also be associated with one or more processes or methods for transmitting module 2050 to computer machine 2000 via network 2080, any signal-bearing medium, or any other communication or transmission technology. Module 2050 may also include hardware circuitry or information for configuring hardware circuitry, such as microcode or configuration information for an FPGA or other PLD.

[0221] Input / output (I / O) interface 2060 can be configured to couple to one or more external devices, receive data from one or more external devices, and send data to one or more external devices. Such external devices, along with various internal devices, can also be referred to as peripheral devices. I / O interface 2060 may include electrical and physical connections that operatively couple various peripheral devices to computing machine 2000 or processor 2010. I / O interface 2060 can be configured to communicate data, address, and control signals between peripheral devices, computing machine 2000, or processor 2010. I / O interface 2060 can be configured to implement any standard interface, such as Small Computer System Interface (“SCSI”), Serial Attached SCSI (“SAS”), Fibre Channel, Peripheral Component Interconnect (“PCI”), High-Speed ​​PCI (“PCIe”), serial bus, parallel bus, Advanced Technology Attachment (“ATA”), Serial ATA (“SATA”), Universal Serial Bus (“USB”), Thunderbolt (“Thunderbolt”), FireWire (“FireWire”), various video buses, etc. I / O interface 2060 can be configured to implement only one interface or bus technology. Alternatively, I / O interface 2060 can be configured to implement multiple interface or bus technologies. I / O interface 2060 can be configured to operate as part of, in whole of, or in conjunction with system bus 2020. I / O interface 2060 may include one or more buffers to buffer transfers between one or more external devices, internal devices, computing machine 2000, or processor 2010.

[0222] The I / O interface 2060 can couple the computing machine 2000 to a variety of input devices, including mice, touch screens, scanners, electronic digitizers, sensors, receivers, touchpads, trackballs, cameras, microphones, keyboards, any other pointing devices, or any combination thereof. The I / O interface 2060 can couple the computing machine 2000 to a variety of output devices, including video displays, speakers, printers, projectors, haptic feedback devices, automatic controls, robotic components, actuators, motors, fans, electromagnets, valves, pumps, transmitters, signal transmitters, lights, etc.

[0223] Computer machine 2000 can operate in a networked environment that uses a logical connection across network 2080 via network interface 2070 to one or more other systems or computers. Network 2080 may include a wide area network (WAN), local area network (LAN), intranet, Internet, wireless access network, wired network, mobile network, telephone network, optical network, or a combination thereof. Network 2080 may be packet-switched, circuit-switched, have any topology, and can use any communication protocol. Communication links within network 2080 may involve various digital or analog communication media, such as fiber optic cables, free-space optics, waveguides, electrical conductors, wireless links, antennas, radio frequency communications, etc.

[0224] Processor 2010 may be connected via system bus 2020 to other components of computing machine 2000, or to the various peripheral devices discussed herein. It should be understood that system bus 2020 may be within processor 2010, outside processor 2010, or both. According to some embodiments, processor 2010, other components of computing machine 2000, or any of the various peripheral devices discussed herein may be integrated into a single device, such as a system-on-a-chip (“SOC”), system-on-package (“SOP”), or ASIC device.

[0225] In systems discussed herein that collect or utilize personal information about users, users may be given the opportunity or option to control whether a program or feature collects user information (e.g., information about the user's social networks, social actions or activities, occupation, hobbies, or current location), or to control whether and / or how content is received from a content server that is more relevant to the user. Additionally, some data may be processed in one or more ways before storage or use, resulting in the removal of personally identifiable information. For example, user identity may be processed to make it impossible to determine the user's personally identifiable information, or, where location information (such as city, zip code, or state level) is available, the user's geographic location may be generalized to make the user's specific location undeterminable. Therefore, users can control how information about themselves is collected and how content servers use that information.

[0226] Embodiments may include a computer program embodying the functionality described and illustrated herein, wherein the computer program is implemented in a computer system including instructions stored in a machine-readable medium and a processor executing the instructions. However, it should be understood that many different ways may exist to implement the embodiments in computer programming, and these embodiments should not be construed as limited to any single set of computer program instructions. Furthermore, skilled programmers will be able to write such computer programs to implement embodiments of the disclosed embodiments based on the accompanying flowcharts and the relevant descriptions in the application text. Therefore, disclosure of a particular set of program code instructions is not considered necessary for a full understanding of how to make and use the embodiments. Furthermore, those skilled in the art will appreciate that one or more aspects of the embodiments described herein can be performed by hardware, software, or a combination thereof, as may be embodied in one or more computing systems. Moreover, any reference to actions performed by a computer should not be construed as being performed by a single computer, as more than one computer may perform the actions.

[0227] The exemplary embodiments described herein can be used with computer hardware and software that perform the methods and processes described herein. The systems, methods, and programs described herein can be embodied in a programmable computer, computer-executable software, or digital circuitry. Software can be stored on a computer-readable medium. For example, computer-readable media may include floppy disks, RAM, ROM, hard disks, removable media, flash memory, memory sticks, optical media, magneto-optical media, CD-ROMs, etc. Digital circuitry may include integrated circuits, gate arrays, building block logic, field-programmable gate arrays (FPGAs), etc.

[0228] The example systems, methods, and actions described in the previously presented embodiments are illustrative, and in alternative embodiments, certain actions may be performed in a different order, performed in parallel with each other, omitted entirely, and / or combined between different example embodiments, and / or certain additional actions may be performed without departing from the scope and spirit of the invention. Therefore, the alternative embodiments are included within the scope of the following claims, which are to be interpreted in the broadest sense to include these alternative embodiments.

[0229] Although specific embodiments have been described in detail above, the description is for illustrative purposes only. Therefore, it should be understood that, unless explicitly stated otherwise, many of the aspects described above are not intended as desired or necessary elements. Without departing from the spirit and scope of the embodiments as defined in the appended claims, modifications and corresponding equivalent components or actions can be made to the disclosed aspects of the exemplary embodiments by those skilled in the art who benefit from this disclosure, in addition to the foregoing. The scope of the appended claims is given the broadest interpretation to cover such modifications and equivalent structures.

Claims

1. A computer-implemented method for identifying a user at a location, comprising: receiving, by one or more processors at a location, one or more face templates from one or more computing devices, each face template associated with a corresponding user associated with a corresponding user computing device that receives a broadcast identifier at the location and retransmits the broadcast identifier to the one or more computing devices, each face template comprising a representation of a respective face image of a respective user; obtaining, by the one or more processors, a face image of a particular user; determining, by the one or more processors, a similarity between the face image of the particular user and each of the one or more face templates; and identifying, by the one or more processors, a particular face template of the one or more face templates as corresponding to the face image of the particular user based on the determined similarity between the face image and the particular face template satisfying a threshold value indicating that the particular face template corresponds to the face image of the particular user.

2. The computer-implemented method of claim 1, wherein: the one or more processors are associated with a merchant system at the location.

3. The computer-implemented method of claim 1, further comprising: generating, by the one or more processors, a face template for the particular user based on the face image.

4. The computer-implemented method of claim 1, further comprising: receiving, by the one or more processors, a request to identify a first user; and transmitting, by the one or more processors, a response to the request to identify the first user, the response comprising an indication of the particular user.

5. The computer-implemented method of claim 4, wherein: the request to identify the first user is received from a merchant system point of sale device; and the response to the request to identify the first user is transmitted to the merchant system point of sale device.

6. The computer-implemented method of claim 5, further comprising: processing, by the merchant system point of sale device, a transaction using payment account information associated with the particular user.

7. The computer-implemented method of claim 1, further comprising: capturing, by one or more image capture devices, a video feed of an environment, wherein the environment comprises the particular user located proximate to one or more merchant system point of sale devices; and extracting, by the one or more processors, the face image of the particular user from the video feed.

8. The computer-implemented method of claim 7, further comprising, by the one or more processors: determining, based on frames of the video feed, a location of a face of the particular user and a location of each of one or more merchant system point of sale devices at a location associated with a merchant system; and ​ ​ assigning the particular user to a particular point of sale device of the one or more merchant system point of sale devices based on a distance between a face of the particular user in the frames of the video feed and each of the one or more merchant system point of sale devices; wherein, the indication of the identity of the particular user further comprises an identifier of the particular point of sale device and an indication that the particular user is assigned to the particular point of sale device.

9. A computer-implemented method of identifying a user at a location, comprising: receiving, by one or more processors, an account identifier of a user and a merchant identifier from a user computing device at a location of a merchant system, wherein the user computing device receives the merchant identifier from a broadcast at the location of the merchant system and retransmits the merchant identifier to the one or more processors; identifying the location of the merchant system associated with the merchant identifier; adding a face template corresponding to the account identifier of the user and the location of the merchant system to a current consumer log comprising face templates of users currently at the location of the merchant system; transmitting the current consumer log to a merchant computing device at the location of the merchant system, wherein the merchant computing device identifies the user based on identifying a face template from the current consumer log that is similar to a face image of the user received by the merchant computing device.

10. The computer-implemented method of claim 9, wherein: the merchant identifier is a merchant beacon identifier received from a merchant beacon device located at the location of the merchant system.

11. The computer-implemented method of claim 9, further comprising receiving, by the one or more processors: determining that the user computing device is no longer proximate to the merchant system; and; removing the face template of the user and the location of the merchant system from the current consumer log.

12. The computer-implemented method of claim 11, wherein, determining that the user computing device is no longer proximate to the merchant system comprises determining that no subsequent transmissions of the account identifier and the merchant identifier have been received from the user computing device within a threshold amount of time.

13. The computer-implemented method of claim 9, further comprising receiving, by the one or more processors: a particular face template associated with the identified user, a transaction total, and a merchant account identifier from a merchant computing device at the location of the merchant system; determining an account identifier associated with the identified user based on the particular face template; transmitting one or more payment account information items associated with the account identifier to a point of sale device; receiving, from the point of sale device, an indication of a selection of a particular payment account information item for use in a transaction; generating a transaction authorization request comprising the transaction total, the merchant account identifier, and the particular payment account information item; transmitting the transaction authorization request to an issuer system associated with the particular payment account information item; receiving, from the issuer system, data including an authorization of the transaction authorization request; and transmitting a message related to the authorization of the transaction to the point of sale device for display on the point of sale device.

14. One or more non-transitory computer-readable media having computer-readable instructions embodied thereon that, when executed by one or more processors, cause the one or more processors to perform operations comprising: receiving, by the one or more processors at a location, one or more face templates from one or more computing devices, each face template associated with a corresponding user associated with a corresponding user computing device that receives a broadcast identifier at the location and retransmits the broadcast identifier to the one or more computing devices, each face template including a representation of a respective face image of a respective user; obtaining a face image of a particular user; determining a similarity between the face image and each of the one or more face templates; and identifying a particular face template of the one or more face templates as corresponding to the face image of the particular user based on the determined similarity between the face image and the particular face template satisfying a threshold value indicating that the particular face template corresponds to the face image of the particular user.

15. The one or more non-transitory computer-readable media of claim 14, wherein: the one or more processors are part of a merchant system at the location.

16. The one or more non-transitory computer-readable media of claim 14, wherein, the operations further comprise: generating a face template for the particular user based on the face image.

17. The one or more non-transitory computer-readable media of claim 14, wherein, the operations further comprise: receiving a request to identify a first user; and transmitting a response to the request to identify the first user, the response including an indication of the particular user.

18. The one or more non-transitory computer-readable media of claim 17, wherein: the request to identify the first user is received from a merchant system point of sale device; the response to the request to identify the first user is transmitted to the merchant system point of sale device; and the operations further comprise processing a transaction by the merchant system point of sale device using payment account information associated with the particular user.

19. The one or more non-transitory computer-readable media of claim 14, wherein, the operations further comprise: determining a location of a face of the particular user and a location of each of one or more point of sale devices at a location associated with a merchant system based on a frame of a video feed; and assigning the particular user to a particular point of sale device of the one or more point of sale devices based on a distance between the face of the particular user and each of the one or more point of sale devices in the frame of the video feed; wherein the indication of the identity of the particular user further includes an identifier of the particular point of sale device and an indication of the assignment of the particular user to the particular point of sale device.

Images