Systems and methods for guiding card positioning using telephone sensors

Abstract

Systems and methods for guiding card positioning using telephone sensors are disclosed. The position alignment system facilitates positioning the contactless card in a "best position" in a target volume relative to the contactless card reading device. Alignment logic uses information captured from available imaging devices, such as infrared proximity detectors, cameras, infrared sensors, dot matrix projectors, etc., to direct the card to a target location. The captured image information is processed using one or both of a machine learning model and / or synchronous localization and mapping logic to identify card locations, trajectories, and predicted locations. Machine learning techniques may be used to intelligently control and customize trajectory adjustments and cue identifications to customize guidance based on the user's preferences and / or historical behavior. As a result, the speed and accuracy of contactless card alignment are improved, and the received NFC signal strength is maximized, thereby reducing the occurrence of offline transactions.

Description

[0001] This application is a divisional application of the Chinese patent application with application number "202080019925.4", application date "July 10, 2020", and title "System and method for guiding card positioning using telephone sensors". Background Art

[0002] Near-field communication (NFC) comprises a set of communication protocols that enable electronic devices, such as mobile devices and contactless cards, to wirelessly transfer information. NFC devices can be used in contactless payment systems, similar to those used by contactless credit cards and electronic ticketing smart cards. For example, in addition to payment systems, NFC-enabled devices can also function as electronic identification documents and key cards.

[0003] For example, contactless devices (e.g., cards, tags, transaction cards, etc.) can use NFC technology to conduct two-way or one-way contactless short-range communication based on, for example, radio frequency identification (RFID) standards, EMV standards, or using, for example, NFC data exchange format (NDEF) tags. The communication can use magnetic field induction to enable communication between powered electronic devices, including mobile wireless communication devices, and unpowered or passively powered devices (such as transaction cards). In some applications, high-frequency wireless communication technology enables data exchange between devices at short distances (such as only a few centimeters), and the two devices can operate most efficiently in certain placement configurations.

[0004] While the advantages of using an NFC communication channel for contactless card transactions are numerous, including simple setup and lower complexity, one difficulty facing NFC data exchanges can be the difficulty of transmitting signals between devices (including contactless cards) with small antennas. During an NFC exchange, movement of the contactless card relative to the device can undesirably affect the NFC signal strength received at the device and interrupt the exchange. Additionally, characteristics of the card (e.g., a metal card) can cause noise, other reflections that inhibit signal reception, or falsely trigger an NFC read transaction. For systems that use contactless cards for authentication and transaction purposes, delays and interruptions can result in lost transactions and customer dissatisfaction. Summary of the Invention

[0005] A system of one or more computers can be configured to perform specific operations or actions by installing software, firmware, hardware, or a combination thereof on the system, which, when in operation, causes the system to perform the actions. One or more computer programs can be configured to perform specific operations or actions by including instructions that, when executed by a data processing device, cause the device to perform the actions.

[0006] According to one general aspect, a method for guiding the positioning of a card to a target position relative to a device includes the steps of: detecting proximity of a card to the device via a proximity sensor; in response to the card approaching the device, the device capturing a series of images of a three-dimensional volume proximate to the device; processing the series of images to determine a position and trajectory of the card within the three-dimensional volume proximate to the device; predicting a projected position of the card relative to the device based on the position of the card and the trajectory of the card; identifying one or more differences between the projected position and the target position, including identifying at least one trajectory adjustment predicted to reduce the one or more differences and one or more prompts predicted to implement the trajectory adjustment; displaying one or more prompts on a display of the device; repeating the steps of: capturing the series of images, determining the position and trajectory of the card, predicting the projected position of the card, identifying one or more differences, at least one trajectory adjustment, and one or more prompts, and displaying the one or more prompts until the one or more differences are within a predetermined threshold; and in response to the one or more differences being within the predetermined threshold, triggering an event at the device to retrieve data from the card. Other embodiments of this aspect include corresponding computer systems, apparatuses, and computer programs recorded on one or more computer storage devices, each configured to perform the actions of the method.

[0007] Implementations may include one or more of the following features. In the method, the step of processing the series of images to determine the position and trajectory of the card within a three-dimensional volume proximate to the device utilizes at least one of a machine learning model or a simultaneous localization and mapping (SLAM) process. The method includes the steps of: during an event, repeating the steps of capturing the series of images, determining the position and trajectory of the card, predicting a projected position of the card, identifying one or more discrepancies, at least one trajectory adjustment, and one or more prompts, and displaying one or more prompts to ensure the discrepancies remain within a predetermined threshold, enabling the device to read data from the card. In the method, the step of triggering the event includes initiating a data exchange between the card and the device, wherein the data exchange is associated with at least one of a financial transaction and an authorization transaction. In the method, the step of capturing the series of images is performed by one or more of a camera of the device, an infrared sensor of the device, or a dot projector of the device, and wherein the series of images includes one or both of two-dimensional image information and three-dimensional image information associated with one or more of infrared energy and visible light energy measured at the device. The method includes the steps of generating a volumetric map of a three-dimensional volume proximate to the device using a series of images obtained from one or more of a camera, an infrared sensor, and a dot projector, the volumetric map comprising pixel data for a plurality of pixel locations within the three-dimensional volume proximate to the device. In the method, processing the series of images to determine the position and trajectory of a card includes forwarding the series of images to a feature extraction machine learning model trained to process the volumetric map to detect one or more features of the card and, responsive to the one or more features, identify the position and trajectory of the card within the volumetric map. In the method, predicting the projected position of the card relative to the device includes forwarding the position and trajectory of the card to a second machine learning model trained to predict the projected position based on historical attempts to locate the card. In the method, the historical attempts used to train the second machine learning model are customized to the user of the device. In the method, the one or more cues include at least one of a visual cue, an audible cue, or a combination of visual and audible cues. Implementations of the described technology may include hardware, methods, or processes, or computer software on a computer-accessible medium.

[0008] According to one general aspect, a device includes a proximity sensor configured to detect whether a card is in proximity to the device; an image capture device coupled to the proximity sensor and configured to capture a series of images of a three-dimensional volume in proximity to the device; a processor coupled to the proximity sensor and the image capture device; a display interface coupled to the processor; a card reader interface coupled to the processor; and a non-transitory medium storing alignment program code configured to guide the card to a target position relative to the device. The alignment program code, when executed by a processor, is operable to: monitor proximity of a card to a device; enable an image capture device to capture a series of images of a three-dimensional volume proximate to the device; process the series of images to determine a position and trajectory of the card within the three-dimensional volume proximate to the device, and predict a projected position of the card relative to the device based on the position of the card and the trajectory of the card; identify one or more differences between the projected position and the target position, including identifying at least one trajectory adjustment and one or more prompts to implement the at least one trajectory adjustment, the at least one trajectory adjustment being predicted to reduce the one or more differences; display one or more prompts on a display interface during at least one of before and during a card reading operation; and trigger a card reading operation via a card reader interface when the one or more differences are within a predetermined threshold. Other embodiments of this aspect include corresponding computer systems, apparatuses, and computer programs recorded on one or more computer storage devices, each configured to perform the actions of the method.

[0009] Implementations may include one or more of the following features. The device of claim 11, wherein the program code uses at least one of a machine learning model or a simultaneous localization and mapping (SLAM) process, the program code being operable, when executed, to process the series of images to determine a position and trajectory of the card within a three-dimensional volume proximate to the device. In the device, the card reading operation is associated with one of a financial transaction and an authorization transaction. In the device, the image capture device includes one or more of a camera, an infrared sensor, or a dot projector, and the series of images captures one or more of infrared energy and visible light energy measured at the device. In the device, the series of images includes one or both of two-dimensional image information and three-dimensional image information. In the device, the alignment program code is further configured to use the series of images, the infrared sensor, and the dot projector to generate a volumetric map of the three-dimensional volume proximate to the device, the volumetric map including pixel data for a plurality of pixel locations within the three-dimensional volume proximate to the device. The device also includes a feature extraction machine learning model trained to locate the card within the three-dimensional volume proximate to the device and to predict a projected position using historical attempts to locate the card. In the device, the historical attempts are user-specific historical attempts. The device wherein the one or more prompts include at least one of a visual prompt, an audible prompt, or a combination of a visual and audible prompt. Implementations of the described technology may include hardware, a method or process, or computer software on a computer-accessible medium.

[0010] According to one general aspect, a method for guiding a card to a target location relative to a device includes the following steps: detecting a request by the device to perform a transaction; measuring the proximity of the card to the device using a proximity sensor of the device; when it is determined that the card is in proximity to the device, controlling at least one of a camera and an infrared depth sensor of the device to capture a series of images of a three-dimensional volume proximate to the device; processing the series of images to determine a position and trajectory of the card in the three-dimensional volume proximate to the device, the processing being performed by at least one of a machine learning model trained using historical attempts to guide the card to the target location or a simultaneous localization and mapping (SLAM) process; predicting a projected position of the card relative to the device based on the position and trajectory of the card; identifying one or more differences between the projected position and the target location, including identifying at least one trajectory adjustment selected to reduce the one or more differences and identifying one or more prompts to implement the trajectory adjustment; displaying one or more prompts on a display of the device; repeating the steps of capturing image information, determining the position and trajectory of the card, predicting the projected position of the card, identifying one or more differences, at least one trajectory adjustment, and one or more prompts, and displaying the one or more prompts until the one or more differences are within a predetermined threshold; and triggering reading of the card by a card reader of the device when the difference is less than the predetermined threshold. Other embodiments of this aspect include corresponding computer systems, apparatus, and computer programs recorded on one or more computer storage devices, each configured to perform the actions of the method. BRIEF DESCRIPTION OF THE DRAWINGS

[0011] Figure 1A and Figure 1B is a schematic diagram provided to illustrate the interaction between a contactless card and a contactless card reading device;

[0012] Figure 2 is a diagram of an exemplary operating volume of a near field communication device;

[0013] Figure 3 is an illustration of a sensor bar of a mobile phone that may be configured to perform positional alignment as disclosed herein;

[0014] Figure 4 is a block diagram illustrating exemplary components of one embodiment of a device configured as disclosed herein;

[0015] Figure 5 It can be done by Figure 4 A flowchart of exemplary steps of a location alignment system and method performed by an NFC transaction device;

[0016] Figure 6 is a detailed flow chart illustrating exemplary steps that may be performed to align the position of a contactless card relative to a device;

[0017] Figure 7is a flowchart illustrating exemplary steps that may be performed to train the machine learning model disclosed herein;

[0018] Figure 8 is a flow chart illustrating exemplary steps that may be performed in a simultaneous localization and mapping (SLAM) process as used as disclosed herein;

[0019] Figure 9 is a flow chart illustrating exemplary steps that may be performed to locate a contactless card for NFC communication using a combination of a proximity sensor and an image capture device of a mobile phone device;

[0020] Figure 10 shows exemplary phone / card interactions and displays during proximity sensing;

[0021] Figure 11 shows exemplary phone / card interactions and displays during position alignment;

[0022] 12A to 12C shows an exemplary mobile phone display that may be provided after successful alignment for NFC communication, including a prompt for adjusting contactless card positioning to maximize signal strength received by the mobile device;

[0023] Figure 13A 、 Figure 13B and Figure 13C shows an exemplary phone / card interaction as disclosed herein; and

[0024] Figure 14 is a flow chart of one embodiment of an exemplary process for controlling an interface of a card reader of a device using captured image data as disclosed herein. DETAILED DESCRIPTION

[0025] The position alignment systems and methods disclosed herein facilitate positioning of a contactless card relative to a device, such as positioning the contactless card close to a target position within a three-dimensional target volume. In one embodiment, the position alignment system uses a proximity sensor of the device to detect the approach of a contactless card. Upon detecting the approach, a series of images may be captured by one or more imaging elements of the device, including, for example, by a camera of the device and / or by an infrared sensor / dot projector of the device. The series of images may be processed to determine the position and trajectory of the card relative to the device. The position and trajectory information may be processed by a predictive model to identify trajectory adjustments to reach the target position and one or more prompts for implementing the trajectory adjustments. This arrangement uses the existing imaging capabilities of the mobile device to provide real-time positioning assistance feedback to the user, thereby improving the speed and accuracy of contactless card alignment and maximizing the received NFC signal strength.

[0026] According to one aspect, a trigger system can automatically initiate near-field communication between a device and a card to transmit a password from the card's applet to the device. The trigger system can operate in response to a darkness level or a change in darkness level in a series of images captured by the device. The trigger system can operate in response to a complexity level or a change in complexity level in a series of images. The trigger system can automatically trigger an operation controlled by the device's user interface, such as automatically triggering the reading of a card. The trigger system can be used alone or in conjunction with one or more aspects of the positional alignment system disclosed herein.

[0027] These and other features of the present invention will now be described with reference to the accompanying drawings, in which like reference numerals are used throughout to refer to like elements. The following detailed description may be presented in terms of program processes executed on a computer or computer network, with general reference to the symbols and terminology used herein. These process descriptions and representations are used by those skilled in the art to most effectively convey the substance of their work to others skilled in the art.

[0028] A process is herein and generally considered to be a self-consistent sequence of operations leading to a desired result. The process can be implemented in the form of hardware, software, or a combination thereof. These operations are those requiring physical manipulation of physical quantities. Typically, although not necessarily, these quantities take the form of electrical, magnetic, or optical signals capable of being stored, transferred, combined, compared, and otherwise manipulated. Mainly for general reasons, it sometimes proves convenient to refer to these signals as bits, values, elements, symbols, characters, terms, numbers, etc. It should be noted, however, that all of these and similar terms are associated with the appropriate physical quantities and are merely convenient labels applied to these quantities.

[0029] Furthermore, the manipulations performed are often referred to by terms such as addition or comparison, which are commonly associated with mental operations performed by a human operator. In any of the operations described herein that form part of one or more embodiments, such capabilities of a human operator are not required, or in most cases, desirable. Rather, these operations are machine operations. Useful machines for performing the operations of the various embodiments include general-purpose digital computers or similar devices.

[0030] Various embodiments also relate to apparatus or systems for performing these operations. This apparatus may be specially constructed for the desired purpose, or it may comprise a general-purpose computer selectively activated or reconfigured by a computer program stored in the computer. The processes presented herein are not inherently related to a particular computer or other apparatus. Various general-purpose machines may be used with programs written in accordance with the teachings herein, or it may prove convenient to construct more specialized apparatus to perform the required method steps. The required structure for the various machines will appear from the description given.

[0031] In the following description, for purposes of explanation, numerous specific details are set forth in order to provide a thorough understanding thereof. However, it will be apparent that the novel embodiments can be practiced without these specific details. In other instances, well-known structures and devices are shown in block diagram form to facilitate description thereof. The intention is to cover all modifications, equivalents, and alternatives consistent with the claimed subject matter.

[0032] Figure 1A and Figure 1B Each shows a mobile phone device 100 and a contactless card 150. The contactless card 150 may include a payment card or transaction card (hereinafter referred to as a transaction card) issued by a service provider, such as a credit card, debit card, or gift card. In some examples, the contactless card 150 is not related to a transaction card and may include, but is not limited to, an identification card or a passport. In some examples, the transaction card may include a dual-interface contactless transaction card. The contactless card 150 may include a substrate comprising a single layer or one or more laminated layers composed of plastic, metal, or other materials.

[0033] In some examples, the contactless card 150 may have physical characteristics that conform to the ID-1 format of the ISO / IEC 7810 standard, and the contactless card may additionally conform to the ISO / IEC 14443 standard. However, it should be understood that the contactless card 150 according to the present disclosure may have different characteristics, and the present disclosure does not require that the contactless card be implemented in a transaction card.

[0034] In some embodiments, the contactless card may include an embedded integrated circuit device that can store, process, and transmit data to another device (such as a terminal or mobile device) via NFC. Common uses of contactless cards include transit tickets, bank cards, and passports. Contactless card standards cover various types included in ISO / IEC 10536 (close-coupled cards), ISO / IEC 14443 (proximity cards), and ISO / IEC 15693 (vicinity cards), each of which is incorporated herein by reference. Such contactless cards are intended to operate very close to, very close to, and at a greater distance from an associated coupling device, respectively.

[0035] Exemplary proximity contactless cards and communication protocols that benefit from the location assistance systems and methods disclosed herein include the proximity contactless cards and communication protocols described in U.S. patent application serial number 16 / 205,119, entitled “Systems and Methods for Cryptographic Authentication of Contactless Cards,” filed by Osborn et al. on November 29, 2018 (hereinafter referred to as the '119 application), and incorporated herein by reference.

[0036] In one embodiment, the contactless card includes an NFC interface composed of hardware and / or software configured for two-way or one-way contactless short-range communication based on, for example, the Radio Frequency Identification (RFID) standard, the EMV standard, or using an NDEF tag. Communication can use magnetic field induction to achieve communication between electronic devices (including mobile wireless communication devices). Short-range high-frequency wireless communication technology enables data to be exchanged between devices within a short distance (such as only a few centimeters).

[0037] NFC uses electromagnetic induction between two loop antennas when NFC-enabled devices exchange information. ISO / IEC 14443-2:2016 (incorporated herein by reference) specifies the characteristics of power and two-way communication between a proximity coupling device (PCD) and a proximity card or object (PICC). The PCD generates a high-frequency alternating magnetic field. This field inductively couples to the PICC to transfer power and is modulated for communication, operating in the 13.56 MHz radio frequency ISM band at rates ranging from 106 to 424 kbit / s over the ISO / IEC 18000-3 air interface. As specified in the ISO standard, PCD transmissions generate a uniform field strength ("H") that varies from an Hmin of at least 1.5 A / m (rms) to an Hmax of 7.5 A / m (rms) to support Class 1, Class 2, and / or Class 3 antenna designs for PICC devices.

[0038] exist Figure 1A and Figure 1B , mobile phone 100 is a PCD device, and contactless card 150 is a PICC device. During a typical contactless card communication exchange, as Figure 1AAs shown, the user can be prompted by the mobile phone 100 to engage the card with the mobile device, for example by including a prompt 125 on the display 130 indicating where the card is to be placed. For the purposes of this application, "engaging" the card with the device includes, but is not limited to, bringing the card into the spatial operating volume of the NFC reading device (i.e., the mobile phone 100), wherein the operating volume of the NFC reading device includes a spatial volume near, adjacent to, and / or surrounding the NFC reading device within which the uniform field strength of the signals transmitted by and between the mobile device 100 and the card 150 is sufficient to support data exchange. In other words, the user can engage the contactless card with the mobile device by tapping the card against the front of the device or holding the card within a certain distance from the front of the device that allows NFC communication to occur. Figure 1A In FIG. 1 , prompt 125 provided on display 130 is provided to achieve this result. Figure 1B The card is shown placed within the transaction volume. Figure 1B During the illustrated transaction, reminder prompts, such as prompt 135, may be displayed to the user.

[0039] An exemplary exchange between phone 100 and card 150 may include activation of card 150 by the RF operating field of phone 100, transmission of a command by phone 100 to card 150, and transmission of a response by card 150 to phone 100. Some transactions may use several such exchanges, and some transactions may be performed by a mobile device using a single read operation of the transaction card.

[0040] In one example, it will be appreciated that successful data transfer is best achieved by maintaining magnetic field coupling to a level at least equal to a minimum (1.5 A / m (rms)) magnetic field strength throughout the transaction, and that magnetic field coupling is a function of signal strength and distance between the card 150 and the mobile phone 100. When testing the compatibility of an NFC-enabled device, e.g., to determine whether the device's power requirements (determines the operating volume), transmission requirements, receiver requirements, and signal form (time / frequency / modulation characteristics) conform to ISO standards, a series of test transmissions are conducted at test points within the operating volume defined by the NFC Forum simulation specification.

[0041] Figure 2 An exemplary operating volume 200, identified by the NFC analog forum, for use when testing NFC-enabled devices is shown. Operating volume 200 defines a three-dimensional volume positioned around a contactless card reader device (e.g., a mobile phone device) and may represent a preferred distance for near field communication exchanges (e.g., for NFC reading a card by the device). To test an NFC device, received signals may be measured at various test points, such as point 210, to verify uniform field strength within the minimum and maximum ranges for the NFC antenna class.

[0042] Although the NFC standard specifies a specific operating volume and test method, it will be readily understood that the principles described herein are not limited to an operating volume having a specific size, and the method does not require that the operating volume be determined based on the signal strength of any particular protocol. Design considerations (including but not limited to the power of the PCD device, the type of PICC device, the expected communication between the PCD and PICC device, the duration of communication between the PCD and PICC device, the imaging capabilities of the PCD device, the expected operating environment of the device, the historical behavior of the device user, etc.) can be used to determine the operating volume used herein. Thus, any discussion below refers to a "target volume," which, in various embodiments, may include the operating volume or a subset of the operating volume.

[0043] Although Figure 1A and Figure 1B In the example, the placement of the card 150 on the phone 100 may appear simple and straightforward, but often the only feedback provided to the user when the card alignment is less than optimal is a failed transaction. A contactless card EMV transaction may include a series of data exchanges requiring connectivity for up to two seconds. During such a transaction, the user juggles the card, and the NFC reader and any merchandise may have difficulty positioning and maintaining the card in the desired position relative to the phone to maintain the preferred distance for a successful NFC exchange.

[0044] According to one aspect, to overcome these problems, a card alignment system and method activates an imaging component of a mobile device to capture a series of images. The series of images can be used to locate the position and trajectory of the card in real time to guide the card to a preferred distance and / or target location for an NFC exchange. The series of images can also be used to automatically trigger an NFC exchange or operation, for example, by measuring darkness levels and / or complexity levels or patterns thereof in the series of captured images.

[0045] For example, using this information, the alignment method can determine a trajectory adjustment and identify a prompt associated with the trajectory adjustment for guiding the card to the target volume. The trajectory adjustment prompt can be presented to the user using the audio and / or display components of the phone to guide the card to the target location within the target volume and / or initiate an NFC read. In various embodiments, the "target location" (or "target positioning") can be defined at various granularities. For example, the target location can include the entire target volume or a subset of the target volume. Alternatively, the target location can be associated with a specific location of the contactless card within the target volume and / or the space surrounding and including the specific location.

[0046] Figure 33 is a front-facing top portion 300 of one embodiment of a mobile phone that can be configured to support the alignment systems and methods disclosed herein. The phone is shown as including a sensor panel 320 disposed along the top edge of portion 300, but it is understood that many devices may include fewer or more sensors that may be positioned differently on their devices, and the present invention is not limited to any particular type, number, arrangement, location, or design of sensors. For example, most phones have a front-facing camera and a front-facing camera and / or other sensors, any of which may be used for the purposes described herein for positional alignment guidance.

[0047] Sensor panel 320 is shown to include infrared camera 302 , flood illuminator 304 , proximity sensor 306 , ambient light sensor 308 , speaker 310 , microphone 312 , front-facing camera 314 , and dot projector 316 .

[0048] Infrared camera 302 can be used in conjunction with dot projector 316 for depth imaging. The infrared emitter of dot projector 316 can project up to 30,000 dots in a known pattern onto an object, such as a user's face. The dots are captured by dedicated infrared camera 302 for depth analysis. Flood illuminator 304 is a light source. Proximity sensor 306 is a sensor capable of detecting the presence of nearby objects without any physical contact.

[0049] Proximity sensors are commonly used in mobile devices and operate to lock out UI input, such as detecting (and skipping) accidental touchscreen taps when the mobile phone is held to the ear. An exemplary proximity sensor operates by emitting an electromagnetic field or a beam of electromagnetic radiation (e.g., infrared) toward a target and measuring the reflected signal received from the target. The design of the proximity sensor can vary depending on the composition of the target; capacitive proximity sensors or photoelectric sensors can be used to detect plastic targets, and inductive proximity sensors can be used to detect metal targets. It should be understood that other methods of determining proximity are within the scope of the present disclosure, and the present disclosure is not limited to proximity sensors that operate by emitting an electromagnetic field.

[0050] The top portion 300 of the phone is also shown to include an ambient light sensor 308 for controlling, for example, the brightness of the phone's display. A speaker 310 and a microphone 312 implement basic phone functions. A front camera 314 can be used for two-dimensional and / or three-dimensional image capture, as described in more detail below.

[0051] Figure 44 is a block diagram of representative components of a mobile phone or other NFC-enabled device incorporating elements that facilitate card position alignment as disclosed herein. These components include interface logic 440, one or more processors 410, memory 430, display control 435, network interface logic 440, and sensor control 450 coupled via a system bus 420.

[0052] Each of the components uses hardware, software, or a combination thereof to perform a specific function. One or more processors 410 may include various hardware elements, software elements, or a combination of the two. Examples of hardware elements may include devices, logic devices, components, processors, microprocessors, circuits, processor circuits, circuit elements (e.g., transistors, resistors, capacitors, inductors, etc.), integrated circuits, application specific integrated circuits (ASICs), programmable logic devices (PLDs), digital signal processors (DSPs), field programmable gate arrays (FPGAs), application-specific standard products (ASSPs), system-on-a-chip systems (SOCs), complex programmable logic devices (CPLDs), memory cells, logic gates, registers, semiconductor devices, chips, microchips, chipsets, etc. Examples of software elements may include software components, programs, applications, computer programs, application programs, system programs, software development programs, machine programs, operating system software, middleware, firmware, software modules, routines, subroutines, functions, methods, programs, processes, software interfaces, application program interfaces (APIs), instruction sets, computing codes, computer codes, code segments, computer code segments, words, values, symbols, or any combination thereof. Determining whether an embodiment is implemented using hardware elements and / or software elements can vary according to many factors, such as desired computing rates, power levels, thermal tolerances, processing cycle budgets, input data rates, output data rates, memory resources, data bus speeds, and other design or performance constraints, as desired for a given implementation.

[0053] The image processor 415 can be any processor, or alternatively can be a dedicated digital signal processor (DSP) for performing image processing on data received from one or more cameras 452, infrared sensor controller 455, proximity sensor controller 457, and dot projector controller 459. The image processor 415 can even utilize SIMD (Single Instruction Multiple Data) or MIMD (Multiple Instruction Multiple Data) technology for parallel computing to improve speed and efficiency. In some embodiments, the image processor can include a system on a chip with a multi-core processor architecture to achieve high-speed, real-time image processing capabilities.

[0054] Memory 430 may include computer-readable storage media to store program code (such as alignment unit program code 432 and payment processing program code 433) and data 434. Memory 430 may also store user interface program code 436. User interface program code 436 may be configured to interpret user input received at user interface elements, including physical elements (such as a keyboard and touch screen 460). User interface program code 436 may also interpret user input received from graphical user interface elements (such as buttons, menus, icons, labels, windows, widgets, etc.) that may be displayed on a user display under the control of display control 435. According to one aspect, and as described in more detail below, memory 430 may also store trigger program code 431. Trigger program code 431 may be used to automatically trigger NFC communication between the device and the card, for example, in response to a determined darkness level and / or complexity level of a series of images captured by camera 452 or other sensor device. In some embodiments, the automatically triggered operations may be those typically performed in response to user input, such as automatically triggering a read operation initiated by activating a user interface element (such as a read button provided on a graphical user interface). Automatic triggering reduces delays and inaccuracies associated with using user interface elements to control NFC communications.

[0055] Examples of computer-readable storage media may include any tangible media capable of storing electronic data, including volatile or non-volatile memory, removable or non-removable memory, erasable or non-erasable memory, writable or rewritable memory, etc. The program code may include executable computer program instructions implemented using any suitable type of code, such as source code, compiled code, interpreted code, executable code, static code, dynamic code, object-oriented code, visual code, etc. The embodiments may also be at least partially implemented as instructions contained in or on a non-transitory computer-readable medium, which may be read and executed by one or more processors to enable performance of the operations described herein.

[0056] The alignment unit program code 432 includes program code for location assistance for contactless card / phone communication disclosed herein. The alignment unit program code 432 can be used by any service provided by the phone that uses contactless card exchange for authentication or other purposes. For example, a service implemented in the payment processing program code 433 (such as a payment processing service) can use contactless card exchange for authentication during the initial stages of a financial transaction.

[0057] The system bus 420 provides an interface for system components including, but not limited to, the memory 430 and the processor 410. The system bus 420 may be any of several types of bus structures that may also interconnect to a memory bus (with or without a memory controller), a peripheral bus, and a local bus using any of a variety of commercially available bus architectures.

[0058] The network interface logic includes transmitters, receivers, and controllers configured to support various known protocols associated with different forms of network communication. Example network interfaces that may be included in a mobile phone implementing the methods disclosed herein include, but are not limited to, a WIFI interface 442, an NFC interface 444, a Bluetooth interface 446, and a cellular interface 448.

[0059] The sensor control 450 includes a subset of sensors that can support the position alignment method disclosed herein, including one or more cameras 452 (which may include camera technology for capturing two-dimensional and three-dimensional light-based images or infrared images), an infrared sensor 454 and an associated infrared sensor controller 455, a proximity sensor 456 and an associated proximity sensor controller 457, and a dot projector 458 and an associated dot projector controller 459.

[0060] Now refer to Figure 5, a flow chart illustrating an exemplary process 500 for contactless card positioning using image information obtained in real time from a sensor of an NFC reading device. The process includes detecting the proximity of a contactless card at step 510, and upon detection, triggering image capture using the device's imaging capabilities at step 515, and processing the captured series of images at step 520. Processing the images may be performed at least in part by alignment unit program code and may include positioning the contactless card within a target volume in proximity to the device and determining the card's trajectory at step 525. Processing the images may also include predicting a trajectory adjustment for aligning the card with a target location within the target volume, identifying a prompt for implementing the trajectory adjustment, and displaying the prompt on the device at step 535. The prompt may include one or more of an instruction (in the form of text or symbols), an image (including one or more of the captured images), a color, a color pattern, a sound, and other mechanisms.

[0061] The process of capturing an image at 515 and processing the image at 520 continues until it is determined at step 540 that the contactless card is in its target position (and / or within a preferred distance from the device). The alignment process may then initiate or cause initiation of a data exchange transaction / communication between the card and the device at step 545. For example, the alignment process may include providing a display prompt to the user to initiate one or more of the transactions. Alternatively, the alignment process may automatically initiate the data exchange process when alignment is detected at step 540. In embodiments utilizing NFC interface technology, the alignment process may open the NFC interface to enable NFC communication and perform NFC communication at step 550.

[0062] Figure 6 FIG6 is a flow chart of a first exemplary embodiment of a position alignment process 600 that uses a machine learning predictive model to process captured images to extract features, position a card within a three-dimensional target volume, and determine a card trajectory. The system may also use the machine learning predictive model to identify trajectory adjustments to move the card to a target position within the target volume and to identify cues to implement the trajectory adjustments.

[0063] In step 605, the phone monitors the reflected energy emitted by the device and reflected back to the device, including detecting the proximity of the card to the device when the reflected energy exceeds a threshold of the proximity sensor. In some phones, the proximity sensor can be implemented using a light sensor chip. Common light sensor chips include the ISL29003 / 23 and GP2A from Intersil and Sharp, respectively. These two sensor chips are primarily active light sensors that provide ambient light intensity in units of LUX. This sensor is implemented as a Boolean sensor. Boolean sensors return two values: "near" and "far." The threshold is based on the LUX value; that is, the LUX value of the light sensor is compared to the threshold. A LUX value exceeding the threshold means that the proximity sensor returns "far." Any value below the threshold means that the sensor returns "near." The actual value of the threshold is conventionally defined and depends on the sensor chip used and its light response, the chip's position and orientation on the smartphone body, the composition and reflective response of the target contactless card, and other factors.

[0064] In step 610, in response to the card approaching the device, the device initiates image capture. Image capture may include capturing a two-dimensional image using one or more cameras accessible on the device. The two-dimensional image may be captured by one or both of a visible light and infrared camera. For example, some mobile devices may include a rear-facing camera capable of capturing high dynamic range (HDR) photos.

[0065] Some mobile devices may include dual cameras that capture images along different imaging planes to create a depth-of-field effect. Some may further include a "selfie" infrared camera or may include infrared emitter technology, such as for projecting dots of infrared light onto a target in a known pattern. These dots can be captured by the infrared camera for analysis.

[0066] The captured images from any one or more of the above sources, and / or subsets or various combinations of the captured images, may then be forwarded to steps 615 and 620 for image processing and contactless card positioning, including determining the position and trajectory of the contactless card.

[0067] According to one aspect, image processing includes constructing a volumetric map of a target volume proximate to the phone, including an area proximate to and / or including at least a portion of an operating volume of an NFC interface of the phone, wherein the volumetric map is represented as a three-dimensional array of voxels storing values ​​associated with the color and / or intensity of the voxels within the visible or infrared spectrum. In some embodiments, a voxel is a discrete element in an array of elements that constitute a volume of a conceptual three-dimensional space, such as each of the array of discrete elements into which a representation of a three-dimensional object is divided.

[0068] According to one aspect, position alignment includes processing voxels of the target volume to extract features of the contactless card to determine the card's position within the target volume, and comparing voxels of the target volume constructed at different time points to track the card's movement over time to determine the card's trajectory. Various processes can be used to track position and trajectory, including using machine learning models and alternatively using SLAM technology, each of which is now described in more detail below.

[0069] Machine learning is a branch of artificial intelligence that is concerned with mathematical models that can learn from data, classify data, and make predictions about data. Such mathematical models (which may be referred to as machine learning models) can classify input data between two or more categories; cluster input data between two or more groups; predict outcomes based on input data; identify patterns or trends in input data; identify the distribution of input data in space; or any combination of these. Examples of machine learning models may include (i) neural networks; (ii) decision trees, such as classification trees and regression trees; (iii) classifiers, such as naive deviance classifiers, logistic regression classifiers, ridge regression classifiers, random forest classifiers, least absolute shrinkage and selector (LASSO) classifiers, and support vector machines; (iv) clusterers, such as k-means clusterers, mean shift clusterers, and spectral clusterers; (v) decomposers, such as decomposition machines, principal component analyzers, and kernel principal component analyzers; and (vi) ensembles or other combinations of machine learning models. In some examples, the neural network can include a deep neural network, a feedforward neural network, a recurrent neural network, a convolutional neural network, a radial basis function (RBF) neural network, an echo state neural network, a long short-term memory neural network, a bidirectional recurrent neural network, a gated neural network, a layered recurrent neural network, a random neural network, a modular neural network, a spiking neural network, a dynamic neural network, a cascaded neural network, a neuro-fuzzy neural network, or any combination of these.

[0070] Different machine learning models can be used interchangeably to perform tasks. Examples of tasks that can be performed, at least in part, using machine learning models include various types of scoring; bioinformatics; cheminformatics; software engineering; fraud detection; customer segmentation; generating online recommendations; adaptive websites; determining customer lifetime value; search engines; serving ads in real time or near real time; classifying DNA sequences; affective computing; performing natural language processing and understanding; object recognition and computer vision; robotic locomotion; playing games; optimization and metaheuristic algorithms; detecting network intrusions; medical diagnosis and monitoring; or predicting when an asset (such as a machine) will require maintenance.

[0071] A machine learning model can be built by an at least partially automated (e.g., with little or no human involvement) process called training. During training, input data can be iteratively supplied to the machine learning model so that the machine learning model can identify patterns associated with the input data or identify relationships between the input data and output data. Using training, the machine learning model can be transformed from an untrained state to a trained state. The input data can be split into one or more training sets and one or more validation sets, and the training process can be repeated multiple times. The splitting can follow a k-fold cross-validation rule, a leave-one-out rule, a leave-one-out rule, or a leave-one-out rule.

[0072] According to one embodiment, a machine learning model may be trained to identify features of a contactless card using image information captured by one or more imaging elements of an NFC reading device when the card is in proximity to the card, and the feature information may be used to identify the card's location and trajectory within a target volume.

[0073] Now refer to Figure 7 A flowchart 700 provides an overview of a method for training and using a machine learning model for location and trajectory identification. At block 704, training data may be received. In some examples, the training data may be received from a remote or local database, constructed from various data subsets, or input by a user. The training data may be used in its original form to train the machine learning model, or pre-processed into another form before it can be used to train the machine learning model. For example, the original form of the training data may be smoothed, truncated, aggregated, clustered, or otherwise manipulated into another form before it can be used to train the machine learning model. In embodiments, the training data may include communication exchange information, historical communication exchange information, and / or information related to communication exchanges. The communication exchange information may be for a general population and / or specific to users and user accounts in a financial institution's database system. For example, for location alignment, the training data may include processing image data of contactless cards in different weather conditions and from different viewing angles to learn voxel values ​​for card features at these orientations and viewing angles. For trajectory adjustment and cue identification, this training data may include data related to the effect of trajectory adjustment on the card at different locations. The machine learning model may be trained to identify cues by measuring the effectiveness of the cues in achieving trajectory adjustments, where in one embodiment, effectiveness may be measured by time to card alignment.

[0074] At block 706, the training data may be used to train a machine learning model. The machine learning model may be trained in a supervised, unsupervised, or semi-supervised manner. In supervised training, each input in the training data may be associated with a desired output. The desired output may be a scalar, a vector, or a different type of data structure, such as text or an image. This may enable the machine learning model to learn a mapping between the input and the desired output. In unsupervised training, the training data includes the input but not the desired output, so that the machine learning model must find structure in the input on its own. In semi-supervised training, only some of the inputs in the training data are associated with the desired output.

[0075] At block 708, the machine learning model may be evaluated. For example, an evaluation dataset may be obtained, such as through user input or from a database. The evaluation dataset may include inputs related to desired outputs. The inputs may be provided to the machine learning model, and the outputs from the machine learning model may be compared to the desired outputs. If the outputs from the machine learning model closely correspond to the desired outputs, the machine learning model may have high accuracy. For example, if 90% or more of the outputs from the machine learning model are the same as the desired outputs (e.g., current communication exchange information) in the evaluation dataset, the machine learning model may have high accuracy. Otherwise, the machine learning model may have lower accuracy. The 90% figure may be just an example. Realistic and desired accuracy percentages may depend on the problem and the data.

[0076] In some examples, if the machine learning model has insufficient accuracy for the particular task, the process can return to block 706 where the machine learning model can be further trained using additional training data or otherwise modified to improve accuracy. If the machine learning model has sufficient accuracy for the particular task, the process can continue to block 710.

[0077] At this point, one or more machine learning models have been trained using the training dataset to: process the captured images to determine position and trajectory, predict the projected position of the card relative to the device based on the current position and trajectory, identify at least one trajectory adjustment, and implement one or more prompts for the trajectory adjustment.

[0078] At block 710, new data is received. For example, during the position alignment process for each contactless card communication exchange, new data may be received. At block 712, the trained machine learning model may be used to analyze the new data and provide results. For example, the new data may be provided as input to the trained machine learning model. As new data is received, the results of feature extraction predictions, position, and trajectory predictions may be continuously adjusted to minimize the duration of the alignment process.

[0079] In block 714, the results can be post-processed. For example, the results can be added to other data as part of a job, multiplied with other data, or combined in other ways. As another example, the results can be converted from a first format (such as a time series format) to another format (such as a count series format). During post-processing, any number of operations and combinations of operations can be performed on the results.

[0080] Simultaneous localization and mapping (SLAM) is well defined in the field of robotics for dynamic reconstruction of 3D image space. For example, Davidson et al., "MonoSLAM: Real-Time Single Camera SLAM," IEEE Transactions on Pattern Analysis and Machine Intelligence, Vol. 29, No. 6, 2007 (incorporated herein by reference) focuses on localization and presents a real-time algorithm that can recover the 3D trajectory of a monocular camera moving rapidly through a previously unknown scene. According to one aspect, it is recognized that the techniques for camera tracking described by Davidson can be used in the position alignment systems and methods disclosed herein. As described above, SLAM techniques can be used to track the advancement of a phone's camera relative to detected features of a card, rather than tracking the advancement of the card relative to the phone, to achieve a similar result of positioning the card relative to the phone.

[0081] Now refer to Figure 8 , a flowchart showing exemplary steps of a MonoSLAM method 800 for contactless card positioning will now be described, which may be used to perform Figure 6 The functionality of steps 615 and 620 is described in detail. The technique disclosed by Davidson constructs a persistent map of scene landmarks that is referenced indefinitely in a state-based framework. Forming a persistent map may be advantageous when camera motion is restricted, and thus SLAM technology may be advantageous for focusing on the position alignment process of a specific object (such as a contactless card). The use of a persistent map keeps the processing requirements of the algorithm bounded and allows for continuous real-time operation.

[0082] SLAM allows for dynamic probabilistic estimation of the state of a moving camera and its map to constrain predictive search using estimates of motion, thereby guiding efficient processing.

[0083] At step 810, an initial probabilistic feature-based map can be generated that represents a snapshot of the state of the camera and the current estimates of all features of interest at any moment in time, along with the uncertainties in those estimates. The map can be initialized at system startup and persist until the end of operation, but can continuously and dynamically evolve as it is updated with new image information over time. Estimates of the probabilistic state of the camera and features are updated during relative camera / card motion and feature observation. As new features are observed, the map is zoomed in with the new state, and features can be deleted if necessary. However, it should be understood that once features of a contactless card can be identified with high probability certainty, further image processing can limit subsequent searches to the located features.

[0084] The probabilistic nature of the map lies not only in the spread of the average "best" estimates of the camera / card state over time, but also in the first-order uncertainty distribution that describes the magnitude of possible deviations from these values. Mathematically, the map can be represented by a state vector and a covariance matrix P. The state vector x^ can be constructed from the stacked state estimates of the cameras and features, and P can be a square matrix of equal dimensions that can be divided into submatrix elements as shown in Equation 1 below:

[0085] Equation 1:

[0086]

[0087] The probability distribution of all map parameters is approximated as a single multivariate Gaussian distribution in a space with a dimension equal to the size of the total state vector. Specifically, the state vector xv of a camera consists of a metric 3D position vector rW, orientation quaternion qRW, velocity vector vW, and angular velocity vector ωR relative to a fixed world frame W carried by the camera and a “robot” frame R (13 parameters), as shown in the following equation II:

[0088] Equation II:

[0089]

[0090] The characteristic state y i is the 3D position vector of the location of the point feature; according to one aspect, the point feature can include a feature of a contactless card. The role of map 825 allows real-time localization to capture a sparse set of high-quality landmarks. Specifically, it can be assumed that each landmark corresponds to a well-positioned point feature in 3D space. The camera can be modeled as a rigid body, requiring translation and rotation parameters to describe its position, and we also maintain estimates of its linear and angular velocities. According to one aspect, the camera modeling herein can be translated relative to the extracted features (i.e., the contactless card) to define the translation and rotation parameters of the card movement, thereby maintaining the linear and angular velocities of the card relative to the phone.

[0091] In one embodiment, in step 830, Davison uses relatively large (11×11 pixel) image patches as long-term landmark features. Camera positioning information can be used to improve matching over camera displacement and rotation. Salient image regions can initially be automatically detected (i.e., based on card properties) using techniques such as those described in J. Shi and C. Tomasi, “Good Features to Track”, Proc. IEEE Conf. Computer Vision and Pattern Recognition, pp. 593-600, 1994 (incorporated herein by reference) (which provides repeatable visual landmark detection). Once the 3D positions (including depth) of the features are fully initialized, each feature can be stored as an oriented planar texture. When the feature is measured from a new (relative) camera position, its patch can be projected from 3D to the image plane to produce a template for matching with the real image. The saved feature template is retained over time to enable the position of the feature to be remeasured over an arbitrarily long period of time, thereby determining the feature trajectory.

[0092] According to one embodiment, a constant velocity, constant angular velocity model can be used, which assumes that the camera moves at a constant velocity at all times, with undetermined accelerations occurring within a Gaussian distribution. While this model imparts a certain smoothness to the relative card / camera motion, it imparts robustness to systems using sparse visual measurements. In one embodiment, the predicted position of an image feature (i.e., the predicted card position) can be determined before searching for the feature within the SLAM map.

[0093] One aspect of Davison's method involves predicting feature locations at 850 and restricting image review to the predicted feature locations. Feature matching between the image frames themselves can be performed using a direct normalized cross-correlation search of a template patch projected into the current camera estimate; the template can be scanned across the image and tested for a match starting from the predicted location until a peak is found. Reasonable confidence bounds assume that the image processing work is focused, and restricting the search to a small search region of the input image using a sparse map enables image processing to be performed in real time at high frame rates.

[0094] In one embodiment, predicting the position can be done as follows. First, using the estimate of the camera position x v and the estimated feature position y i , the position of the point feature relative to the camera is expected to be as shown in the following equation III:

[0095] Equation III:

[0096]

[0097] With a perspective camera, we can find the location (u, v) where we expect to find a feature in the image using the standard pinhole model shown in Equation IV below:

[0098]

[0099] where fk u 、fk v , u0, and v0 comprise standard camera calibration parameters. This approach enables active control of the viewing direction towards favorable measurements with high innovation covariance, thus enabling limiting the maximum number of feature searches per frame to the 10 or 12 most informative searches.

[0100] According to one aspect, it will therefore be appreciated that the performance benefits associated with SLAM, including the ability to perform real-time localization of contactless cards while limiting extraneous image processing, will benefit the position registration system disclosed herein.

[0101] Back to Figure 6 Once position and trajectory information is available through a machine learning model, SLAM technology, or other methods, according to one aspect, the position alignment system and method includes a process 625 for predicting trajectory adjustments and associated prompts to guide the card to the target position within the target volume. According to one aspect, the prediction can be performed using a predictive model (such as a machine learning model trained and maintained using the above-described machine learning principles) to identify trajectory adjustments and prompts based on the effectiveness of previous trajectory adjustments and prompts, and the prediction is thereby customized by the user's behavior. Trajectory adjustments can be determined, for example, by identifying the variance between the target position and the predicted position and selecting adjustments to the current trajectory to minimize the variance. Effectiveness can be measured in a variety of ways, including but not limited to the duration of the position alignment process. For example, in some embodiments, the artificial intelligence, neural network, or other aspects of the machine learning model can autonomously select those prompts that are most effective in helping the user achieve the final result of card alignment.

[0102] In some embodiments, it is contemplated that a trajectory adjustment can be linked to a set of one or more prompts configured to implement the associated trajectory adjustment. The set of one or more prompts may include auditory and visual prompts and may be in the form of one or more instructions (in text or symbolic form), images (including one or more of captured images), colors, color patterns, sounds, and other mechanisms displayed by the device. In some embodiments, an effectiveness value may be stored for each prompt, wherein the effectiveness value is related to historical reactions and the effectiveness of such prompt in displaying the implementation of the trajectory adjustment. The effectiveness value may be used by a machine learning model to select one or more of the trajectory adjustments and / or prompts to guide the card to the target location.

[0103] At step 630, a prompt may be displayed on the phone's display. At step 635, the process continues to capture image information, determine location and trajectory, identify trajectory adjustments, and display prompts until, at step 635, it is determined that the difference between the target location and the predicted location is within a predetermined threshold. The predetermined threshold is a matter of design choice and may vary depending on one or more of the target volume, the NFC antenna, and the like.

[0104] Once it is determined at step 635 that the variance is within the threshold, the card may be deemed aligned at step 630 and the NFC mobile device may be triggered at step 640 to initiate a communication exchange with the card.

[0105] According to one aspect, the data exchange may be a cryptographic data exchange as described in the '119 application. During the cryptographic exchange, after communication has been established between the phone and the contactless card, the contactless card may generate a message authentication code (MAC) cryptogram according to the NFC data exchange format. In particular, this may occur when reading (such as an NFC read) a Near Field Data Exchange (NDEF) tag, which may be created according to the NFC data exchange format. For example, by device 100 ( Figure 1A ) can send a signal to the contactless card 150 ( Figure 1A ) transmits a message, such as an applet selection message (with the applet ID of an NDEF generation applet), where the applet may be an applet stored in the memory of the contactless card and operable to generate an NDEF tag when executed by the processing component of the contactless card. After confirming the selection, a sequence of a select file message followed by a read file message may be transmitted. For example, the sequence may include "select function file," "read function file," and "select NDEF file." At this point, a counter value maintained by the contactless card may be updated or incremented, which may be followed by "read NDEF file."

[0106] At this point, a message can be generated, which can include a header and a shared secret. A session key can then be generated. A MAC password can be created from the message, which can include the header and the shared secret. The MAC password can then be concatenated with one or more random data blocks, and the MAC password and random number (RND) can be encrypted with the session key. Thereafter, the password and header can be concatenated, encoded as ASCII hexadecimal, and returned in the NDEF message format (in response to the "Read NDEF File" message).

[0107] In some examples, the MAC password may be transmitted as an NDEF tag, and in other examples, the MAC password may be included with a uniform resource indicator (eg, as a formatted string).

[0108] In some examples, the application may be configured to transmit a request to the contactless card, the request including instructions to generate a MAC password, and the contactless card sends the MAC password to the application.

[0109] In some examples, the transmission of the MAC password occurs via NFC, however, the present disclosure is not limited thereto. In other examples, this communication can be carried out via Bluetooth, Wi-Fi or other wireless data communication methods.

[0110] In some examples, a MAC password can function as a digital signature for authentication purposes. For example, in one embodiment, the MAC password can be generated by a device configured to implement key diversification using a counter value. In such a system, the transmitting device and the receiving device can be provided with the same master symmetric key. In some examples, the symmetric key can include a shared secret symmetric key that can be kept secret from all parties except the transmitting and receiving devices involved in exchanging secure data. It should also be understood that both the transmitting and receiving devices can be provided with the same master symmetric key, and that a portion of the data exchanged between the transmitting and receiving devices includes at least a portion of data that can be referred to as a counter value. The counter value can include a number that changes each time data is exchanged between the transmitting and receiving devices. Furthermore, the transmitting and receiving devices can use a suitable symmetric cryptographic algorithm, which can include at least one of a symmetric encryption algorithm, an HMAC algorithm, and a CMAC algorithm. In some examples, the symmetric algorithm used to process the diversification value can include any symmetric cryptographic algorithm that can be used to generate a diversified symmetric key of a desired length. Non-limiting examples of symmetric algorithms may include symmetric encryption algorithms (such as 3DES or AES128), symmetric HMAC algorithms (such as HMAC-SHA-256 algorithm), and symmetric CMAC algorithms (such as AES-CMAC).

[0111] In some embodiments, the transmitting device can employ a selected cryptographic algorithm and use a master symmetric key to process the counter value. For example, a sender can select a symmetric encryption algorithm and use a counter that is updated with each conversation between the transmitting and receiving devices. The transmitting device can then use the master symmetric key to encrypt the counter value using the selected symmetric encryption algorithm, thereby creating a diversified symmetric key. The diversified symmetric key can be used to process sensitive data before transmitting the result to the receiving device. The transmitting device can then transmit the protected encrypted data, along with the counter value, to the receiving device for processing.

[0112] The receiving device can first obtain the counter value and then perform the same symmetric encryption using the counter value as the input for encryption and the master symmetric key as the key for encryption. The output of the encryption can be the same diversified symmetric key value created by the sender. The receiving device can then obtain the protected encrypted data and decrypt the protected encrypted data using a symmetric decryption algorithm and the diversified symmetric key to reveal the original sensitive data. The next time sensitive data needs to be sent from the sender to the receiver via the corresponding transmission device and receiving device, a different counter value can be selected to generate a different diversified symmetric key. By processing the counter value using the master symmetric key and the same symmetric cryptographic algorithm, both the transmitting device and the receiving device can independently generate the same diversified symmetric key. This diversified symmetric key (instead of the master symmetric key) can be used to protect sensitive data.

[0113] In some examples, the key diversification value may include a counter value. Other non-limiting examples of key diversification values ​​include: a random number generated each time a new diversified key is needed and sent from the transmitting device to the receiving device; the full value of a counter value sent from the transmitting device and the receiving device; a portion of a counter value sent from the transmitting device and the receiving device; a counter maintained independently by the transmitting device and the receiving device but not sent between the two devices; a one-time passcode exchanged between the transmitting device and the receiving device; and a cryptographic hash of sensitive data. In some examples, one or more portions of the key diversification value may be used by each party to create multiple diversified keys. For example, a counter may be used as the key diversification value. Further, a combination of one or more of the above-described exemplary key diversification values ​​may be used.

[0114] Figure 9900 is a flowchart illustrating the use of the position alignment system disclosed herein to align a contactless card with an NFC mobile device equipped with a proximity sensor and imaging hardware and software. At step 905, the position alignment logic detects a request by the device to perform a communication exchange. At step 910, the position alignment logic (using the device's proximity sensor) measures the reflected energy emitted by the device and reflected back to the device, including determining when the reflected energy exceeds a predetermined threshold indicating proximity of the card to the device.

[0115] Figure 10 A contactless card 1030 is shown near the operating volume 1020 of the proximity sensor 1015 of the phone 1010. When the phone enters the operating volume 1020, in one embodiment, an infrared beam emitted by the proximity sensor 1015 is reflected back to the proximity sensor 1015 as a signal R 1035. As the card moves closer to the operating volume of the phone, the reflected signal strength increases until a trigger threshold is reached, at which point the proximity sensor indicates the card is "near." In some embodiments, during the proximity search, the phone's display 1050 can display information such as by providing a Figure 10 It is shown as a notification that it is searching for cards, prompting the user by providing visual or auditory instructions, etc.

[0116] In step 915 ( Figure 9 ), when the proximity sensor is triggered, the location alignment logic controls at least one of the device's camera and infrared depth sensor to capture a series of images of a three-dimensional volume proximate to the device when reflected energy exceeds a predetermined threshold. Depending on the location of the NFC reader and the location of the camera on the phone, it will be appreciated that a camera including an operating volume that overlaps at least a portion of the operating volume of the phone's NFC interface may be selected for image capture.

[0117] At step 920, positional alignment logic processes the captured multiple images to determine the position and trajectory of the card in a three-dimensional volume proximate to the device. As previously described, this processing can be performed by one or both of a machine learning model trained using historical attempts to guide the card to a target location and a simultaneous localization and mapping (SLAM) process. At step 925, the positional alignment process predicts a projected position of the card relative to the device based on the position and trajectory of the card, and at step 930, identifies one or more differences between the projected position and the target position, including identifying at least one trajectory adjustment selected to reduce the one or more differences, and identifying one or more cues to implement the trajectory adjustment, and at step 935, the positional alignment process displays the one or more cues on a display of the device.

[0118] Figure 11An exemplary display 1105 of a phone 1110 is shown capturing image information associated with a card 1150 within a target volume 1120. The display 1105 may include a plurality of cues, such as a location cue 1115 associated with a target location, an image cue 1130, and an arrow cue 1140 that may be displayed to a user to assist in directing the card 1150 to the target location. The image cue 1130 may include, for example, a portion of an image captured by an imaging component of the phone 1110 during position alignment and may be beneficial to the user in helping the user understand their movement relative to the target. The arrow 1140 may provide directional assistance, such as Figure 11 As shown, the user is monitored for adjusting the card upwards for proper alignment. Other types of prompts may also be used, including but not limited to text instructions, symbols and / or emoticons, auditory instructions, and color-based guidance (i.e., showing a first color (such as red) to the user when the card is relatively far from the target, and turning the screen green when the card becomes aligned).

[0119] In step 940 ( Figure 9 ), the position alignment process may repeat the following steps until one or more differences are within a predetermined threshold: capturing image information, determining the position and trajectory of the card, predicting the projected position of the card, identifying one or more differences, at least one trajectory adjustment, and one or more prompts, and displaying one or more prompts. At step 945, if the difference is less than the predetermined threshold, the position alignment process may trigger a card reader of the device to read the card. In some embodiments, the position alignment process may continue to operate during data exchange between the card and the mobile device, for example, providing a prompt to adjust the card's position if the card moves during reading.

[0120] Figure 12A 、 Figure 12B and Figure 12C is an example of a display prompt that may be provided by the position alignment process once alignment is detected. Figure 12A In the embodiment of the present invention, when the card is aligned with the target position, a prompt 1220 can be provided to notify the user. In some embodiments, the interface can provide a link (such as link 1225) to enable the user to initiate a card read via the phone. In other embodiments, alignment can automatically trigger a card read.

[0121] exist Figure 12B In the embodiment of the present invention, during the card reading process, prompts may be provided to the user, such as a countdown prompt 1230. In addition, additional prompts, such as arrows 1240, may be provided to enable the user to correct any movement that may occur to the card during reading to ensure that connectivity is not lost and to increase the success rate of NFC communication. Figure 12C As shown, after reading, the display provides a notification 1250 to the user regarding the success or failure of the communication exchange.

[0122] Thus, there have been shown and described a position alignment system and method that facilitates positioning a contactless card at a preferred location relative to a contactless card reading device within a target volume. Alignment logic uses information captured from available imaging devices, such as infrared proximity detectors, cameras, infrared sensors, dot projectors, and the like, to guide the card to the target location. The captured image information can be processed using one or both of a machine learning model and / or simultaneous positioning and mapping logic to identify the card position, trajectory, and predicted location. Machine learning techniques can be used to intelligently control and customize trajectory adjustments and prompt identification to customize guidance based on user preferences and / or historical behavior. As a result, the speed and accuracy of contactless card alignment are improved, and the received NFC signal strength is maximized, thereby reducing the occurrence of dropped transactions.

[0123] The above technology has discussed various methods for placing a contactless card in a desired position relative to a card reader interface of a device once the proximity of the card is initially detected using a proximity sensor. However, it will be understood that the principles disclosed herein can be extended to use captured image data to detect the proximity of the card, thereby enhancing or completely replacing the proximity sensor information. The captured image information can also be processed to determine when the card is in a particular position relative to the card reader interface and automatically perform operations associated with user interface elements, such as automatically triggering an NFC read operation or other function by a mobile device without waiting for user input. This arrangement enables the ability to automatically trigger control operations without the need for user input (e.g., bypassing the need for human interaction with a user interface element of the device).

[0124] According to one aspect, the image processing logic 415 ( Figure 4 ) can be enhanced to include program code for determining image parameters that can indicate the proximity of a card to a card reader. For example, the image parameters can be related to proximity characteristics of the image (i.e., characteristics that indicate that an object may be close to the camera). In some embodiments, the card reader can be located on the same surface as the camera of the device used to capture the image, and thus the image information can further indicate the proximity of the card to the card reader. In various embodiments, the card reader / camera can be located on the front or back of the device.

[0125] In some embodiments, the image parameters include one or more of the darkness level and / or complexity level of the image. For example, referring now briefly to Figure 13A and Figure 13B, device 1310 can be a device having a contactless card reading interface configured as described above to obtain a MAC password from a contactless card 1320, for example, when the card 1320 is brought into proximity with the device 1310. For example, the device can send an applet selection message with an applet ID of an NDEF generation applet, where the applet can be an applet stored in a memory of the contactless card and operable to generate an NDEF tag when executed by a processing component of the contactless card. According to one aspect, a series of images can be captured using a camera of the device, and darkness levels and / or complexity levels can be analyzed to determine when the card is likely to be at a preferred distance from the device, thereby automatically triggering the forwarding of an NFC read operation from the contactless card's NDEF generation applet.

[0126] exist Figure 13A and Figure 13B 13. In FIG. 13, image 1320 is shown on display 1340 of device 1310 for purposes of explanation only, although it is not necessary to display a captured image on device 1310 for determining card proximity as disclosed herein.

[0127] According to one embodiment, when a device initiates NFC communication (e.g., by a user selecting an NFC read operation (such as button 1225) on a user interface on the device, or by the device receiving a request from a third party (such as a merchant application or mobile communication device) for the device to initiate NFC communication with a card, etc.), the device may capture a series of images of a spatial volume proximate to the device. The series of images may be processed to identify one or more image parameters of one or more of the images in the series, including but not limited to a darkness level or complexity level of the image. The complexity level and / or darkness level may be used to trigger an NFC read. Alternatively, or in combination, the image processing may include identifying trends and / or patterns in darkness and / or complexity levels of the series of images or portions of the series of images that indicate the advancement of a card. Identification of trends and / or patterns within the series of images that indicate a card may be at a preferred distance relative to the device may be used to automatically trigger an NFC read.

[0128] For example, 13A to 13C As shown, when the card is farther from the device, the image captured (here represented as image 1330A) may be relatively brighter than image 1330B captured relatively later when the card 1320 is closer to the device. Figure 13B As the card moves closer, the image becomes darker until, as shown in Figure 13C As shown, the captured image (in Figure 13C The card 1320 blocks light from appearing in the image. This may be because the card (or hand) may block ambient light received by the camera when the card is close to the device.

[0129] As mentioned above, the presence of a card at a preferred distance from the device can be determined based on the darkness level, darkness level trend, complexity level, and / or complexity level trend in a series of captured images. Specifically, the presence of a card can be determined by processing pixel values ​​from the series of images to identify the darkness level of each processed pixel. For example, a grayscale value can be assigned to each pixel. The darkness level of the image can be determined by averaging the darkness levels of the image pixels. In some embodiments, when the card is at the preferred distance from the device, the darkness level can be compared to a threshold corresponding to a darkness level, such as a distance that supports a successful NFC read operation. In some embodiments, the threshold can be an absolute threshold; for example, in a system where "0" indicates white and "1" indicates black, a card can be considered "present" and the reader can be enabled when the darkness level is 0.8 or greater. In other embodiments, the threshold can be a relative threshold that takes into account the ambient light of the environment in which the communication exchange will occur. In such embodiments, the first captured image can provide a baseline darkness level, and the threshold can be related to the amount by which the threshold is exceeded to trigger NFC communication; for example, the threshold can be a relative threshold. For example, in a dark room with an initial darkness level of 0.8, it may be desirable to delay triggering NFC communication until the darkness level equals 0.95 or higher.

[0130] In addition to triggering NFC communication based on individually calculated darkness levels, the system also contemplates identifying trends or patterns in image darkness levels to trigger NFC readings. Identifying trends can include, for example, determining an average over a collection of images and triggering a read when the average over the collection of images meets a threshold. For example, while a single image may exceed the threshold, the card's position may not be stable enough to perform an NFC read, and it may be desirable to require a predetermined number of consecutively captured images to exceed the darkness threshold before triggering a read. Additionally or alternatively, consecutively processed images can be monitored to identify spikes and / or plateaus—that is, sudden shifts in darkness levels that persist between consecutive images, indicating activity at the reader.

[0131] In some embodiments, the darkness level of the entire image can be determined by averaging at least a subset of the calculated pixel darkness values. In some embodiments, certain darkness values ​​can be weighted to increase their relevance to the darkness level calculation; for example, portions of the image known to be near a reader or closer to an identified feature may be weighted more highly than portions farther from the reader.

[0132] As described above, a complexity level may be calculated for each captured image, where the complexity level is generally related to the frequency distribution of pixel values ​​within the captured image. In one embodiment, the complexity value may be determined pixel by pixel by comparing the pixel value of each pixel with the pixel values ​​of one or more adjacent pixels. As the card gets closer to the device, such as Figure 13B As shown, if the card is positioned correctly, the background image may be obscured by the card. When the card covers the image, the image becomes more uniform by default, and adjacent pixels generally contain the same pixel value. In various embodiments, the complexity can be determined for each pixel in the image or for a subset of pixels at previously identified locations in the image. The complexity of each pixel can be determined by examining the adjacent pixel values. The complexity level of the entire image can be determined by averaging at least a subset of the calculated pixel complexity values. In some embodiments, certain complexity levels can be weighted to increase their relevance to the complexity calculation; for example, those portions of the image that are known to be close to the card reader or identified feature may have a higher weight than those portions that are farther away from the card reader or identified feature.

[0133] In other embodiments, machine learning methods such as those disclosed herein can enhance image processing, for example by identifying patterns in pixel darkness / pixel complexity values ​​in successive images that indicate known card activity approaching a card reader. Such patterns can include, for example, pixel darkness / complexity levels that vary in a known manner (i.e., darkening from top to bottom or from bottom to top). Patterns can also include image elements that aid in card recognition (such as stripes, icons, printing, etc.) and can be used as described above to provide cues for correct placement of a particular identified card. Over time, information associated with successful and unsuccessful card reads can be used to determine an appropriate image pattern that establishes card presence for a successful NFC card communication exchange.

[0134] Figure 14 14 is a flow chart of exemplary steps that may be used to trigger NFC card reading using one or both of the above-mentioned darkness and / or complexity image attributes. In step 1410, near field communication may be initiated by the device. The initiation of near field communication may be due to the selection of a user interface element on the device (such as Figure 12A Alternatively, or in combination, initiation of near field communication may occur as a result of an action by an application executing on the device (e.g., an application utilizing a password from a card for authentication or other purposes).

[0135] During initiation of NFC communication, at step 1420, a camera of the device (such as a front-facing camera) may capture a series of images of a volume of space in front of the device's camera. In some embodiments, 60, 120, 240, or more images may be captured per second, although the present disclosure is not limited to capturing any particular number of images in the series. At step 1430, the images may be processed to identify one or more image parameters, such as a darkness level representing the distance between the card and the device. At step 1440, the processed darkness level of the image is compared to a predetermined darkness level (e.g., a darkness level associated with a preferred distance for near field communication operations). At step 1450, when the darkness is determined to correspond to the preferred darkness level for an NFC read operation, an NFC read operation may be automatically triggered, for example, to transmit a password from the card's applet.

[0136] In some embodiments, the automatic triggering of the NFC read operation can bypass or replace the triggering historically provided by the user interface element. For example, in some embodiments, a graphical user interface element such as a read button (1225) can be provided on the device so that when the user determines that the card can be properly positioned relative to the device, the user can activate NFC communication. In some embodiments, the user interface element can be associated with a function such as a read operation. It will be understood that the technology described herein can be used to trigger other user interface elements, and various corresponding associated functions can be automatically triggered. The automatic triggering disclosed herein can reduce the delay and inaccuracy associated with the historically controlled user interface elements, thereby improving the NFC communication flow and success rate.

[0137] Thus, a system and method for using captured image information to detect the presence of a card to trigger an NFC read has been shown and described. Such a system can utilize the machine learning methods and / or SLAM methods described in more detail above to provide additional guidance before triggering the card read. With such an arrangement, card placement is improved and the success rate of NFC communication exchanges can be increased.

[0138] As used in this application, the terms "system," "component," and "unit" are intended to refer to a computer-related entity, either hardware, a combination of hardware and software, software, or software in execution, examples of which are described herein. For example, a component can be, but is not limited to, a process running on a processor, a processor, a hard drive, multiple storage drives, non-transitory computer-readable media (of optical and / or magnetic storage media), an object, an executable file, a thread of execution, a program, and / or a computer. For example, both an application running on a server and the server can be components. One or more components can reside in a process and / or thread of execution, and a component can be located on one computer and / or distributed between two or more computers.

[0139] Furthermore, components can be communicatively coupled to each other via various types of communication media to coordinate operations. Coordination can include unidirectional or bidirectional information exchange. For example, components can transmit information in the form of signals transmitted via the communication media. This information can be implemented as signals assigned to various signal lines. In this assignment, each message is a signal. However, other embodiments may alternatively employ data messages. Such data messages can be sent via various connections. Exemplary connections include parallel interfaces, serial interfaces, and bus interfaces.

[0140] Some embodiments may be described using the expression "one embodiment" or "an embodiment" and their derivatives. These terms mean that a particular feature, structure, or characteristic described in connection with the embodiment is included in at least one embodiment. The appearance of the phrase "in one embodiment" in different places in the specification does not necessarily all refer to the same embodiment. In addition, unless otherwise stated, the features described above are considered to be usable together in any combination. Therefore, any features discussed individually may be used in combination with each other, unless it is noted that the features are incompatible with each other.

[0141] With general reference to the symbols and nomenclature used herein, the detailed description herein may be presented in terms of functional blocks or units that can be implemented as program processes executed on a computer or network of computers. These program descriptions and representations are used by those skilled in the art to most effectively convey the substance of their work to others skilled in the art.

[0142] A procedure is here, and generally, conceived to be a self-consistent sequence of operations leading to a desired result. These operations are those requiring physical manipulations of physical quantities. Usually, though not necessarily, these quantities take the form of electrical, magnetic, or optical signals capable of being stored, transferred, combined, compared, and otherwise manipulated. It has proven convenient at times, principally for reasons of common usage, to refer to these signals as bits, values, elements, symbols, characters, terms, numbers, or the like. It should be noted, however, that all of these and similar terms are to be associated with the appropriate physical quantities and are merely convenient labels applied to these quantities.

[0143] Furthermore, the manipulations performed are often referred to by terms such as addition or comparison, which are commonly associated with mental operations performed by a human operator. In any of the operations described herein that form part of one or more embodiments, such capabilities of a human operator are not required, or in most cases, desirable. Rather, these operations are machine operations. Useful machines for performing the operations of the various embodiments include general-purpose digital computers or similar devices.

[0144] Some embodiments may be described using the expressions "coupled" and "connected" and their derivatives. These terms are not necessarily intended to be synonymous with each other. For example, some embodiments may be described using the terms "connected" and / or "coupled" to indicate that two or more elements are in direct physical or electrical contact with each other. However, the term "coupled" may also mean that two or more elements are not in direct contact with each other, but still cooperate or interact with each other.

[0145] It is emphasized that the Abstract of the present disclosure is provided to allow the reader to quickly ascertain the nature of the technical disclosure. It is submitted with the understanding that it will not be used to interpret or limit the scope or meaning of the claims. Furthermore, in the preceding Detailed Description, various features are grouped together in a single example to simplify the disclosure. This disclosure method should not be interpreted as reflecting an intention that the claimed embodiments require more features than expressly recited in each claim. On the contrary, as reflected in the following claims, the inventive subject matter lies in less than all the features of a single disclosed embodiment. Accordingly, the following claims are hereby incorporated into the Detailed Description, with each claim standing on its own as a separate embodiment. In the appended claims, the terms "including" and "in which" are used as the plain-English equivalents of the respective terms "comprising" and "wherein," respectively. Furthermore, the terms "first," "second," "third," etc. are used merely as labels and are not intended to impose numerical requirements on their objects.

[0146] What has been described above includes examples of the disclosed architecture. It is, of course, not possible to describe every conceivable combination of components and / or methodologies, but one skilled in the art will recognize that many further combinations and permutations are possible. Therefore, the novel architecture is intended to embrace all such alterations, modifications, and variations that fall within the spirit and scope of the appended claims.

Claims

1. A computer method comprising: determining, by a processor of the computing device, a series of images of the contactless card; identifying, by the processor, a position and a trajectory of the contactless card from the series of images of the contactless card; predicting, by the processor, a projected position of the card relative to the device based on the position of the card and the trajectory of the contactless card; determining, by the processor, whether the card is in a target position relative to the device; In response to determining that the card is in the target position, triggering, by the processor, an operation of receiving data from the card; and In response to determining that the card is not in the target position: identifying, by the processor, one or more differences between the projected position and the target position, including identifying at least one trajectory adjustment predicted to reduce the one or more differences; determining, by the processor, one or more prompts to implement the trajectory adjustment; as well as The processor causes presentation of the one or more prompts on a display.

2. The computer method of claim 1 , wherein determining the series of images uses at least one of a machine learning model or a simultaneous localization and mapping (SLAM) process.

3. The computer method of claim 1, wherein triggering the operation comprises initiating a data exchange between the card and the computing device.

4. The computer method of claim 1 , comprising capturing the series of images via one or more of a camera of the device, an infrared sensor of the device, or a dot projector of the device.

5. The computer method of claim 4, wherein the series of images includes one or both of two-dimensional image information and three-dimensional image information.

6. The computer method of claim 1 , further comprising: A volumetric map approximates a three-dimensional volume of the computing device using the series of images, the volumetric map comprising pixel data approximates a plurality of pixel locations within the three-dimensional volume of the device.

7. The computer method of claim 6 , wherein identifying the position and trajectory of the card comprises forwarding the series of images to a feature extraction machine learning model trained to process the volumetric map to detect one or more features of the card and to identify the position and trajectory of the card in the volumetric map in response to the one or more features.

8. The computer method of claim 1 , wherein predicting the projected position of the card relative to the device comprises forwarding the position and trajectory of the card to a second machine learning model trained to predict the projected position based on historical attempts to locate the card.

9. The computer method of claim 1, wherein the one or more prompts include at least one of a visual prompt, an audible prompt, or a combination of a visual and audible prompt.

10. The computer method of claim 1, wherein determining whether the card is in a target position relative to the device comprises determining whether the one or more differences are within a predetermined threshold.

11. A computing device comprising: One or more processors configured to: determining a series of images of a contactless card; identifying a location and a trajectory of the contactless card from the series of images of the contactless card; predicting a projected position of the card relative to the device based on the position of the card and the trajectory of the contactless card; determining whether the card is in a target position relative to the device; In response to determining that the card is in the target position, triggering an operation of receiving data from the card; as well as In response to determining that the card is not in the target position: identifying one or more differences between the projected position and the target position, including identifying at least one trajectory adjustment predicted to reduce the one or more differences; determining one or more cues to implement the trajectory adjustment; as well as The one or more prompts are caused to be presented on a display.

12. The computing device of claim 11, wherein the one or more processors are configured to determine the series of images using at least one of a machine learning model or a simultaneous localization and mapping (SLAM) process.

13. The computing device of claim 11, wherein receiving data from the card comprises exchanging data related to at least one of a financial transaction and an authorization transaction.

14. The computing device of claim 11, further comprising an image capture device configured to capture the series of images, wherein the image capture device comprises one or more of a camera, an infrared sensor, or a dot projector.

15. The computing device of claim 14, wherein the series of images includes one or both of two-dimensional image information and three-dimensional image information.

16. The computing device of claim 11, wherein the one or more processors are further configured to generate a volumetric map approximating a three-dimensional volume of the device using the series of images, the volumetric map comprising pixel data approximating a plurality of pixel positions within the three-dimensional volume of the device.

17. The computing device of claim 16, further comprising a feature extraction machine learning model trained to process the volumetric map to detect one or more features of the card and to identify a location and trajectory of the card in the volumetric map in response to the one or more features.

18. The computing device of claim 11, further comprising a second machine learning model trained to predict the projected location based on historical attempts to locate the card.

19. The computing device of claim 11, wherein the one or more prompts include at least one of a visual prompt, an audible prompt, or a combination of a visual and audible prompt.

20. The computing device of claim 11, wherein the one or more processors are configured to determine whether the card is in a target position relative to the device by determining whether the one or more differences are within a predetermined threshold.

Images