Imputing sparse sequence data for recurrent neural network models

Abstract

A computer-implemented method for modelling a sequence of transactions with an imputed data set is provided. The method comprises inputting, by a training module, an input data set into a first neural network, training, by the training module, the first neural network with the input data set, training, by the training module, a second neural network with the input data set and state vectors and generating, by a data imputing module, imputable data. A system for generating an imputed data set from transaction data is also provided.

Description

TITLE IMPUTING SPARSE SEQUENCE DATA FOR RECURRENT NEURAL NETWORKMODELSTECHNICAL FIELD

[0001] The present disclosure is generally related to machine learning techniques to enhance models for sequential transaction data and, more particularly, is directed to techniques for addressing sparsities in transaction data.BACKGROUND

[0002] Machine learning techniques are currently employed to solve many problems in financial technology (or fintech) sector, such as detection of fraudulent transactions. For example, a payment processing network may model a consumer’s spending habits by mapping historical transaction data associated with one or more of payment devices of a user to known output values in a supervised learning environment. In many cases, the models are incorporated into useful tools for classifying and / or approximating measurable attributes of the consumer’s behavior, many of which may then be used to identify future unauthorized and / or fraudulent use of the consumer’s payment devices or the identifiers / credentials associated therewith. Accordingly, maintaining an accurate consumercentric model can provide the consumer with greater protective measures from identity theft.

[0003] Artificial neural networks are powerful computing systems with applications in many areas based on their ability to learn and model non-linearities and complex relationships. In the context of fintech, artificial neural networks have emerged as a useful tool for solving problems involving sequences of data. For example, recurrent neural networks (“RNNs”) are widely adopted in methods for detecting fraudulent activity in transaction data rely. Generally, currently employed RNN models operate on the notion that time intervals between time steps containing transaction data are equal in length and / or uniformly distributed among the transaction data. However, in many instances, transaction datasets may include many time steps where data is absent, otherwise known as sparsities. Furthermore, the locations of the sparsities may not follow any particular pattern or distribution with respect to each other and / or to time steps associated with transaction data. Consequently, training current RNN models with transaction datasets may not produce models which perform as intended and in some instances, the training may fail to converge at all due to computational issues arising from RNN algorithms which are not currently equipped to handle sparsities during training steps. Thus, current approaches to detecting fraudulent transactions may not be able to assess unseen circumstances with sufficient accuracy and / or speed to effectively identify and / or stop fraudulent activity.

[0004] Accordingly, there exists a need for alternative methods and devices for modelling sparse datasets, such as, for example, transaction data sets. The present disclosure provides various solutions that employ a neural network model for addressing sparsities in datasets.SUMMARY

[0005] In various aspects, a computer-implemented method for modelling a sequence of transactions with an imputed data set is disclosed. In some aspects, the method includes inputting, by a training module, an input data set into a first neural network, training, by the training module, the first neural network with the input data set, training, by the training module, a second neural network with the input data set and state vectors, and generating, by a data imputing module, imputable data. In some aspects, the input data set is indexed to a set of time steps including a first subset of time steps and a second subset of time steps. In some aspects, the input data set includes transaction data indexed to the first subset of time steps. In some aspects, the transaction data is associated with the sequence of transactions. In some aspects, the sparsities are indexed to the second subset of time steps. In some aspects, training the first neural network with the input data set generates a first model based on a first set of trained model parameters and state vectors. In some aspects, the state vectors are produced by a transformation, based on the first model, of the input data set. In some aspects, training the second neural network with the input data set and the state vectors generates a second model including a second set of trained model parameters. In some aspects, the second set of trained model parameters are based on a mapping of the input data set to the state vectors. In some aspects, the imputable data is generated with the first set of trained model parameters, the second set of trained model parameters and the state vectors. In some aspects, the imputable data is indexed to the second subset of time steps.

[0006] In various aspects, a system for generating an imputed data set from transaction data is disclosed. In some aspects, the system includes a training module for modelling the transaction data and a data imputing module. In some aspects, the training module includes a first training model and a second training model. In some aspects, the first training model includes a Recurrent Neural Network configured to accommodate sparsities in the transaction data. In some aspects, the first training model is configured to output state vectors and first model parameters based on a training with the transaction data. In some aspects, the second training model includes a fully connected neural network configured to output second model parameters based on a mapping of the transaction data to the state vectors. In some aspects, the data imputing module is for generating new data to satisfy the sparsities. In some aspects, the new data is based on the transaction data, the statevectors, the first model parameters, and the second model parameters.BRIEF DESCRIPTION OF THE DRAWINGS

[0007] In the description, for purposes of explanation and not limitation, specific details are set forth, such as particular aspects, procedures, techniques, etc. to provide a thorough understanding of the present technology. However, it will be apparent to one skilled in the art that the present technology may be practiced in other aspects that depart from these specific details.

[0008] The accompanying drawings, where like reference numerals refer to identical or functionally similar elements throughout the separate views, together with the detailed description below, are incorporated in and form part of the specification, and serve to further illustrate aspects of concepts that include the claimed disclosure and explain various principles and advantages of those aspects.

[0009] The apparatuses, systems, and methods disclosed herein have been represented where appropriate by conventional symbols in the drawings, showing only those specific details that are pertinent to understanding the various aspects of the present disclosure so as not to obscure the disclosure with details that will be readily apparent to those of ordinary skill in the art having the benefit of the description herein.

[0010] FIG. 1 illustrates a payment network environment according to at least one aspect of the present disclosure.

[0011] FIG. 2 shows a block diagram of a training system according to at least one aspect of the present disclosure.

[0012] FIG. 3 illustrates a schematic representation of a training model according to at least one aspect of the present disclosure.

[0013] FIG. 4 is a logic flow diagram of a computer-implemented method for modelling a sequence of transactions with an imputed data set, in accordance with at least one aspect of the present disclosure.

[0014] FIG. 5 is a block diagram of a computer apparatus with data processing subsystems or components, according to at least one aspect of the present disclosure.

[0015] FIG. 6 is a diagrammatic representation of an example system that includes a host machine within which a set of instructions to perform any one or more of the methodologies discussed herein may be executed, according to at least one aspect of thepresent disclosure.

[0016] Corresponding reference characters indicate corresponding parts throughout the several views. The exemplifications set out herein illustrate various aspects of the present disclosure, in one form, and such exemplifications are not to be construed as limiting the scope of the disclosure in any manner.DESCRIPTION

[0017] The following disclosure may provide exemplary systems, devices, and methods for conducting a financial transaction and related activities. Although reference may be made to such financial transactions in the examples provided below, aspects are not so limited. That is, the systems, methods, and apparatuses may be utilized for any suitable purpose.

[0018] Neural networks generally contain interconnected nodes arranged in layers such as input layers, hidden layers and output layers. Each node can process data based on its given activation function which may then be communicated via a connection to one or more nodes in successive layers for further processing. Weights associated with the connections may determine the extent to which an output of a node will influence other nodes. Additionally, a node may bias the sum of its inputs prior to processing. When a neural network is trained with an example via data fed into the input layer, the difference between the processed data obtained at the output layer and expected data from the example may be assessed. Any successive adjustments to the weights are determined according to learning rules with the goal of producing outputs with the neural network which better represent the expected results of the example.

[0019] As described hereinabove, currently employed RNN models, such as, for example, LSTM RNNs, operate on the notion that time intervals between time steps containing transaction data are equal in length and / or uniformly distributed among the transaction data. For example, inputting a sequence of non-zero temporal data into a Long- Short Term Memory (LSTM) RNN will generally produce an accurate model of the data, especially if all neighboring elements are equally spaced with respect to time and / or if the extent of spacing is not representative of any useful information. However, in many instances, spacing between non-zero points may be irregular and meaningful, such as sparsities representing variable gap lengths between transactions in transaction datasets. As used in the present disclosure, sparsities or sparse data may refer to empty or zero-value elements belonging to a minority population of a dataset.

[0020] Furthermore, the locations of the sparsities may not follow any particular patternor distribution with respect to each other and / or to time steps associated with transaction data. Consequently, training current RNN models with transaction datasets may not produce models which perform as intended and in some instances, the training may fail to converge at all due to computational issues arising from RNN algorithms which are not currently equipped to handle sparsities during training steps. Thus, current approaches to detecting fraudulent transactions may not be able to assess unseen circumstances with sufficient accuracy and / or speed to effectively identify and / or stop fraudulent activity.

[0021] The present disclosure provides various solutions that employ a neural network model for addressing sparsities in datasets. For example, in various aspects, the present disclosure provides a system for generating imputed data with a combination of various neural networks including a modified RNN. Additionally, the present disclosure provides a method incorporating the modified RNN to generate and use imputable data to more faithfully represent attributes of training data while avoiding issues associated with sparsities in data sets when developing a predictive model for fraud detection. Thus, various systems and methods of the present disclosure are used to overcome issues with training conventional RNNs even in cases where sparsities in training data may normally be problematic with respect to model training and performance.

[0022] As used herein, the term “system” may refer to one or more computing devices or combinations of computing devices (e.g., processors, servers, client devices, software applications, components of such, and / or the like).

[0023] An “issuer” can include a payment account issuer. The payment account (which may be associated with one or more payment devices) may refer to any suitable payment account (e.g. credit card account, a checking account, a savings account, a merchant account assigned to a consumer, or a prepaid account), an employment account, an identification account, an enrollment account (e.g. a student account), etc.

[0024] As used herein, a “user” may include an individual or a user that may be associated with one or more personal accounts and / or consumer devices. The user may also be referred to as a cardholder, account holder, or consumer.

[0025] The terms “client device” and “user device” refer to any electronic device that is configured to communicate with one or more servers or remote devices and / or systems. A client device or a user device may include a mobile device, a network-enabled appliance (e.g., a network-enabled television, refrigerator, thermostat, and / or the like), a computer, a POS system, and / or any other device or system capable of communicating with a network. A client device may further include a desktop computer, laptop computer, mobile computer(e.g., smartphone), a wearable computer (e.g., a watch, pair of glasses, lens, clothing, and / or the like), a cellular phone, a network-enabled appliance (e.g., a network-enabled television, refrigerator, thermostat, and / or the like), a point of sale (POS) system, and / or any other device, system, and / or software application configured to communicate with a remote device or system.

[0026] As used herein, the term “merchant” may refer to one or more individuals or entities (e.g., operators of retail businesses that provide goods and / or services, and / or access to goods and / or services, to a user (e.g., a customer, a consumer, a customer of the merchant, and / or the like) based on a transaction (e.g., a payment transaction)). As used herein “merchant system” may refer to one or more computer systems operated by or on behalf of a merchant, such as a server computer executing one or more software applications.

[0027] A “payment network” may refer to an electronic payment system used to accept, transmit, or process transactions made by payment devices for money, goods, or services. The payment network may transfer information and funds among issuers, acquirers, merchants, and payment device users.

[0028] As used herein, the term “communication” and “communicate” may refer to the reception, receipt, transmission, transfer, provision, and / or the like of information (e.g., data, signals, messages, instructions, calls, commands, and / or the like). A communication may use a direct or indirect connection and may be wired and / or wireless in nature. As an example, for one unit (e.g., a device, a system, a component of a device or system, combinations thereof, and / or the like) to communicate with another unit means that the one unit is able to directly or indirectly receive information from and / or transmit information to the other unit. The one unit may communicate with the other unit even though the information may be modified, processed, relayed, and / or routed between the one unit and the other unit. In one example, a first unit may communicate with a second unit even though the first unit receives information and does not communicate information to the second unit. For example, a first unit may be in communication with a second unit even though the first unit passively receives data and does not actively transmit data to the second unit. As another example, a first unit may communicate with a second unit if an intermediary unit (e.g., a third unit located between the first unit and the second unit) receives information from the first unit, processes the information received from the first unit to produce processed information, and communicates the processed information to the second unit. In some non-limiting embodiments, a message may refer to a packet (e.g., a data packet, a network packet, and / or the like) that includes data. It will be appreciated that numerous other arrangementsare possible.

[0029] A “communication channel” may refer to any suitable path for communication between two or more entities. Suitable communications channels may be present directly between two entities such as a payment processing network and a merchant or issuer computer, or may include a number of different entities. Any suitable communications protocols may be used for generating a communications channel. A communication channel may in some instances comprise a “secure communication channel” or a “tunnel,” either of which may be established in any known manner, including the use of mutual authentication and a session key and establishment of a secure communications session. However, any method of creating a secure communication channel may be used, and communication channels may be wired or wireless, as well as long-range, short-range, or medium-range. By establishing a secure channel, sensitive information related to a payment device (such as account number, CVV values, expiration dates, etc.) may be securely transmitted between the two entities to facilitate a transaction.

[0030] An “interface” may include any software module configured to process communications. For example, an interface may be configured to receive, process, and respond to a particular entity in a particular communication format. Further, a computer, device, and / or system may include any number of interfaces depending on the functionality and capabilities of the computer, device, and / or system. In some embodiments, an interface may include an application programming interface (API) or other communication format or protocol that may be provided to third parties or to a particular entity to allow for communication with a device. Additionally, an interface may be designed based on functionality, a designated entity configured to communicate with, or any other variable. For example, an interface may be configured to allow for a system to field a particular request or may be configured to allow a particular entity to communicate with the system.

[0031] An “application” or “application program interface” (API) refers to computer code or other data sorted on a computer-readable medium that may be executed by a processor to facilitate the interaction between software components, such as a client-side front-end and / or server-side back-end for receiving data from the client. An “interface” refers to a generated display, such as one or more graphical user interfaces (GUIs) with which a user may interact, either directly or indirectly (e.g., through a keyboard, mouse, touchscreen, etc.).

[0032] As used herein, the term “server” may include one or more computing devices which can be individual, stand-alone machines located at the same or different locations,may be owned or operated by the same or different entities, and may further be one or more clusters of distributed computers or “virtual” machines housed within a datacenter. It should be understood and appreciated by a person of skill in the art that functions performed by one “server” can be spread across multiple disparate computing devices for various reasons. As used herein, a “server” is intended to refer to all such scenarios and should not be construed or limited to one specific configuration. Further, a server as described herein may, but need not, reside at (or be operated by) a merchant, a payment network, a financial institution, a healthcare provider, a social media provider, a government agency, or agents of any of the aforementioned entities. The term “server” may also refer to or include one or more processors or computers, storage devices, or similar computer arrangements that are operated by or facilitate communication and processing for multiple parties in a network environment, such as the Internet, although it will be appreciated that communication may be facilitated over one or more public or private network environments and that various other arrangements are possible. Further, multiple computers, e.g., servers, or other computerized devices, e.g., point-of-sale devices, directly or indirectly communicating in the network environment may constitute a “system,” such as a merchant's point-of-sale system. Reference to “a server” or “a processor,” as used herein, may refer to a previously-recited server and / or processor that is recited as performing a previous step or function, a different server and / or processor, and / or a combination of servers and / or processors. For example, as used in the specification and the claims, a first server and / or a first processor that is recited as performing a first step or function may refer to the same or different server and / or a processor recited as performing a second step or function.

[0033] As used herein, the term “account identifier” may refer to one or more types of identifiers associated with an account (e.g., a unique identifier of an account, an account number, a PAN, a card number, a payment card number, a token, and / or the like) of a user. In some non-limiting embodiments, an issuer may provide an account identifier (e.g., a PAN, a token, a globally unique identifier (GUID), a universally unique identifier (UUID), and / or the like) to a user that uniquely identifies one or more accounts associated with that user. In some non-limiting embodiments, an account identifier may be embodied on a payment device (e.g., a portable financial instrument, a payment card, a credit card, a debit card, and / or the like) and / or may be electronic information communicated to the user that the user may use for electronic payment transactions. In some non-limiting embodiments, an account identifier may be an original account identifier, where the original account identifier was provided to a user at the creation of the account associated with the account identifier. In some non-limiting embodiments, the account identifier may be an account identifier (e.g., a supplemental account identifier) that is provided to a user after the original account identifierwas provided to the user. For example, if the original account identifier is forgotten by the user, stolen from the user, and / or the like, a supplemental account identifier may be provided to the user. In some non-limiting embodiments, an account identifier may be directly or indirectly associated with an issuer such that an account identifier may be a token that maps to a PAN or other type of identifier. Account identifiers may be alphanumeric, any combination of characters and / or symbols, and / or the like.

[0034] FIG. 1 illustrates a payment network environment 100 with which a training system 1000 can interface to model transaction data, according to at least one non-limiting aspect of the present disclosure. The payment network environment 100 depicts, for example, one or more user devices 110n associated with a single entity and / or user, a payment network system 106, an acquirer system 104 and a merchant device 102 associated with a merchant. The payment network system 106 is in communication with the acquirer system 104, and the acquirer system 104 is in communication with the merchant device 102.

[0035] In the illustrated example, the issuer system 108 is in two way communication with each of the user devices 110n and the payment network system 106. Each of the user devices 110n may be identified on the issuer system 108 by a respective account identifier associated with the single entity and / or user and communicate with the issuer system 108 via an issuer provided interface or application over a respective communication channel. Further, each of the account identifiers may include payment credentials. The user devices 11 On can include one or more types of payment devices, such as, for example, a portable financial instrument, a payment card, a credit card, and / or a debit card. While FIG. 1 illustrates two user devices similar in type, the payment network 100 can include any number of additional user devices associated with the same entity and / or user as the illustrated user devices 110n.

[0036] The payment network system 106 may locally store data associated with transactions, such as transactions initiated with one or more of the user devices 110n, occurring within the payment network environment 100. As illustrated in FIG. 1, the training system 1000 is effectively connected to the payment network system 106 such that it may access the transaction data stored by the payment network system 106. Additionally and / or alternatively, the payment network system 106 may store data on a remote server accessible by the training system 1000 via a dedicated communication channel and / or a dedicated session, for example. The transaction data associated with the user devices 11 On may include data representing the account identifiers associated with the user devices 11 On initiating a transaction and time of transaction among other transaction attributes.

[0037] FIG. 2 provides a block diagram of a training system 1000, according to at least one non-limiting aspect of the present disclosure. In various aspects, the training system 1000 includes a training module 1100 and an imputing module 1200. The training module 1100 includes a first training model 1110 and a second training model 1120. As discussed in greater detail below, the first training model 1110 may feed the second training model to facilitate the generation of imputable data by the imputing module 1200. In some aspects, the training system 1000 includes an input module 1001 for inputting training data into the training module 1100. The input module 1001 may include dedicated connections to memory and / or non-volatile storage mediums of the payment network system 106 containing transaction data copied from the payment network system 106. The training system 1000 may also include an output module 1002 for outputting data, such as imputed data generated by the imputing module 1200. Other configurations are contemplated by the present disclosure. For example, in some implementations, the input module 1001 may be incorporated into the training module 1100 and the output module 1002 may be incorporated into the imputing module 1200.

[0038] FIG. 3 illustrates a schematic representation of the first training model 1110 according to one non-limiting aspect of the present disclosure. In various aspects, the first training model 1110 is a neural network containing an input layer 1112, one or more hidden layers 1114 and an output layer 1116. For example, as illustrated in FIG. 3, the first training model 1110 may include 3 hidden layers 1114. However, the number of hidden layers 1114 can be adjusted as desired depending on the training data. The first training model 1110 can accept a training dataset in vector and / or matrix form at the input layer 1112, and produce state vectors in the hidden 1114 and / or output layers 1116 of lower dimension than the input data. In other words, the first training model 1110 produces hidden state and / or output state vectors which convey a simple output, such as a 1-dimensional vector representing a prediction of a valid or a fraudulent transaction, based on a higher dimension input vector which can contain detailed information of the transaction. In some aspects, the input layer 1112 of the first training model 1110 may include a number of nodes corresponding to a number of learnable features in the training data and / or a dimensionality of an input vector containing the training data. Although the illustrated example of the first training model 1110 is limited to 9 nodes in 3 hidden layers 1114, the number of nodes can be adjusted to accommodate a desired number of features. For example, in some implementations, the first training model 1110 can be configured to process a number of learnable features in the range of about 50 to 100 features. In one aspect, the input layer 1112 can separate inputted transaction data associated with a common user and / or entity based on payment device, for example.

[0039] Now referring to FIGs. 2-3, the first training model 1110 is configured to accept temporal data. For example, the first training model 1110 may be initialized to accept an input data set 1101 including input vectors Xtindexed to a set T of time steps tt. Other data configurations are contemplated by the present disclosure. For example, in one implementation, the input layer 1112 of the first training module can be configured to assign indices to raw transaction data.

[0040] In various aspects, the input data set 1101 for the first training model 1110 includes a temporal sequence of valid non-zero transaction data originating from multiple payment devices and / or interfaces linked to a single user / entity. Features of the non-zero transaction data, such as payment device, account identifier, amount, number of line items, purchase category, merchant, and / or payment device location, are represented as discrete elements of the input vectors Xt. In some aspects, the dimension of the input vector defines the number of features therein. In certain aspects, the input data set 1101 is learnable. For example, each of the input vectors Xtcan include features of a common category at a common position, such as in a certain row number. In certain aspects, the input data set 1101 includes learnable data from two or more payment devices of a single user / entity.

[0041] In various aspects, the non-zero transaction data may be separated by sparsities, or empty cells, representative of various gap lengths, where no activity has taken place, between transactions. Alternatively, or additionally, the sparsities may be a result of missing data where a transaction may have occurred but the data associated therewith was not recorded in the data set 1101. In some aspects, a sparsity can be present in the data set 1101 as one or more elements of an input vector Xt. In certain aspects, the data set 1101 can contain one or more input vectors Xtwhich are sparse. In one aspect, the sparsities are not positioned throughout the dataset according to any predetermined distribution with respect to time and / or to any predetermined relationship with transaction data. Other configurations are contemplated by the present disclosure. For example, in some implementations, an input vector can include both non-zero and sparse elements as respective features.

[0042] As described above, the entire data set 1101 can be indexed to a set T of time steps. In various aspects, the non-zero transaction data can be indexed to a first subsetof time steps tj and the sparsities can be indexed to a second subset t2of time steps tj. For descriptive purposes hereinafter, the time steps tj belonging to the first subsetwill be referred to asand the time steps belonging to the second subset ft2will be referred to as In some aspects, an input vector containing only non-zero elements can be indexed tothe first subsetand an input vector containing one or more sparse elements may be referred to as a sparsity which can be indexed to the second subset t2. The input vectors indexed to the first and second subsets may be referred to as, respectively, XtWand Xt^. In one aspect, the first subset of time steps can be the majority group and the second subset of time steps tt2can be a minority group in the set T of time steps, or vice-versa. Other configurations are contemplated by the present disclosure. For example, in some implementations, where input vectors may include both non-zero and sparse elements as respective features, the input vectors may be indexed to time steps on an element-by- element basis.

[0043] Still referring to FIGs. 2-3, In various aspects, one or more of the hidden layers 1114 of the first training model 1110 includes a modified RNN structure 1115 which explicitly includes a term for differentiating various intervals of gap lengths between time steps in training data, such as, for example, the term (t(-in the following equation:where ttis a time of a given event being evaluated by the first training model, tt_ is a time of an event most recently evaluated by the first training model, Xt. is training data of the input vector associated with time St. is a state vector of the modified RNN at current time tbSt._iis a previous output from the modified RNN at time tt-l tand M1represents trained parameters and / or transformation functions of the first training model 1110 which process Xt. and Stto produce StThe state vectors Stcan be defined with a lower dimension than the input vectors. For example, the state vectors Stcan convey a simple output representing a prediction of a valid or a fraudulent transaction based on the detailed information of a higher dimension input vector. In some aspects, the trained model parameters of M1can include filter weights, regression weights, biases and transfer functions such as the sigmoid function and / or the hyperbolic tangent function. In one aspect, the training module 1100 may selectively input members of the first subset of time steps such that all of the state vectors Stproduced during the training of the model 1110 are associated with non-zero data and the parameters of M±are not trained with any input vectors consisting entirely of sparsities. Said in another way, the first training model 1110 can incorporate an undersampling algorithm.

[0044] Still referring to FIG. 2, the second training model 1120 includes a model structure for mapping all of the original training data Xtto the state vectors Stproduced by the first training model 1110. The second training model 1120 can be a fully-connected neural network (FCNN), wherein every node in one layer is fed forward to every node in thenext layer. In some aspects, the FCNN configuration of the second training model 1120 can provide greater computational efficiency than other neural networks, such as, for example, a convolutional neural network. In certain aspects, the second training model 1120 is an FCNN having a structure:Af2(X£) « Stwherein M2represents trained parameters and transformation functions of the second training model 1120, which process Xtto approximate state St. In one aspect, the second training model 1120 maps input vectors having a higher dimension than the state vectors to a lower dimension of the state vectors. After training a model, the training module 1100 may store and / or otherwise preserveand / or M2in an addressable location, such as a nonvolatile computer storage medium and / or addressable memory, which is accessible by the imputing module 1200 for further processing as discussed in greater detail below.

[0045] Still referring to FIG. 2, the imputing module 1200 can utilize the original training data Xtand outputs from the training module 1100 to produce imputable data. In some aspects, an input module of the imputing module 1200 may address memory locations or other data storage locations of the training system 1000 which contain modelsand M2and training data set Xt. In certain aspects, the imputing module 1200 includes processorexecutable instructions. In one aspect, the imputing module 1200 can include a dedicated processor for extracting and / or processing data from the training data set Xtand / or models M and M2.

[0046] Further to the above, the imputing module 1200 includes an imputing algorithm 1210 for generating imputable data X'tto supplement the input data set Xtat locations of sparsities therein. In various aspects, for a given sparsity at a time step tb, the imputing module 1200 generates imputable data X'tbased on previously trained models M1and M2and portions of the input data set Xtclosest to the given sparsity with respect to time. Thus, the imputable data X'tis new data building on the previous executed trainings of the training module 1100 rather than being arbitrarily generated and / or sampled from preexisting data. In some aspects, the imputable data X'tis indexed to a time step tbas it is generated.

[0047] In certain aspects, the imputing algorithm 1210 can identify output data Stassociated with non-zero input data for N number of time steps t“ closest to tband subsequently, generate a system of N number of the following equations:where t' is a time step associated with a given sparsity in input data set Xt, ttis one of the Ntime steps t“ closest to t , St. is the previously determined state vector at time tt, and M1and M2represent the previously trained parameters and transformation functions of the first training model 1110 and the second training model 1120. In one aspect, the N number of time steps t precede tf. Subsequently, the system of N equations can be solved by a processor of the imputing module 1200 to obtain imputable data X'f. Additionally, the N number of closest time steps may be determined according to a number of features and / or a dimension of the input vectors Xf. For example, when a given sparsity is associated with a first payment device of a user, the imputing algorithm can employ data obtained from modelling other payment devices associated with the user, provided that the modeled data falls within the N number of closest time steps.

[0048] In various aspects, the imputing algorithm is executed for each of the sparsities in the original training data set. In some instances where the imputing module 1200 contains multiple processors and / or cores, the imputing algorithm may be executed for each of the sparsities in input data setin parallel. Alternatively, or additionally, the imputing module 1200 may outsource all or computationally intensive portions of the imputing algorithm to a dedicated cluster.

[0049] The equations in the system of N equations generated by the imputing algorithm are similar in nature to the structures of the first and second training models 1110 and 1120. Therefore, the imputable data X'tcan be generated with a high degree of confidence with respect to adhering to the information learned during the training of first and second training models 1110 and 1120. Accordingly, the training system 1000 can generate imputable data which is at least partially representative of a transaction the user may have initiated.Additionally, since the terms (tj - ^') andaccount for specific irregularities in gap lengths between two time steps being considered, the trained model parameters and imputable data generated by the system 1000 can preserve any relevant temporal relationships between neighboring transactions with a higher degree of specificity than conventional RNN-based modelling systems. Thus, the original training data in combination with the imputed data generated by the training system 1000 can be inputted into another RNN model without suffering from poor computational efficiency during training and / or producing models with poor predictive performance. Accordingly, the training system 1000 can generate imputable data to supplement transaction data associated with a user to facilitate the development of predictive models for detecting fraudulent activity in any one of the payment devices linked to the user.

[0050] Further to the above, the imputing module 1200 may store the imputed dataand / or input data Xtfor later use. In some aspects, the imputing module 1200 may incorporate the non-zero elements of the input data Xtand the imputable data X'tinto a new training data set. Alternatively, or additionally, the imputing module 1200 may transmit an imputed training data set to a separate training module such that a new training model for fraud detection is trained on data that is representative of a user’s activities (e.g., spending behavior) without any sparsities therein.

[0051] FIG. 4 is a logic flow diagram of a computer-implemented method 2000 for modelling a sequence of transactions with an imputed data set, which may be carried out using a modelling system such as the training system 1000 of FIG. 2, in accordance with at least one non-limiting aspect of the present. The method 2000 includes inputting 2100, by a training module, an input data set 2001 into a first neural network 2002, training 2200, by the training module, the first neural network 2002 with the input data set 2001 , training 2300, by the training module, a second neural network 2003 with the input data set 2001 and state vectors Stgenerated with the first neural network 2002, and generating 2400, by a data imputing module, imputable data X't. In some aspects, training 2200 the first neural network 2002 generates a first modelincluding a first set of trained model parameters and training 2300 the second neural network 2003 generates a second model Mzincluding a second set of trained model parameters. As described in greater detail below, the method 2000 can facilitate the development of predictive models for detecting fraudulent activity.

[0052] In various aspects, the input data set inputted 2100 by the training module into the first neural network 2002 is indexed to a set of time steps comprising a first subset of time steps and a second subset of time steps. The input data set 2001 includes sequential transaction data comprising learnable features, indexed to the first subset of time steps, and sparsities representing an absence of a transaction event and / or missing transaction data, indexed to the second subset of time steps. In some aspects, the input data set 2001 includes input vectors Xtor matrices comprising the transaction data. The input vectors Xtcan include learnable transaction features arranged into discrete elements of an input vector. In certain aspects, the input data set 2001 is transaction data associated with a single user and / or entity. In one aspect, the input data set 2001 is transaction data associated with multiple payment devices linked to a single user. Further, the first neural network 2002 is a RNN having a structure similar to the modified RNN structure 1115. Thus, in some aspects, the first neural network 2002 can be trained with the transaction data to abstract qualities of a user’s characteristic spending behaviors even if the transaction data is compiled from transactions originating from different payment devices and / or transactions separated in time by varying gap lengths.

[0053] In various aspects, training 2300 the second neural network 2003 generates a second model M2including a second set of trained model parameters. In some aspects, the second neural network is a FCNN which can be configured similarly to the second training model 1120 of FIG. 2. Thus, the training 2300 can generate the second model M2based on a mapping of the input data set 2001 to the state vectors Stgenerated by the training 2200 of the first neural network 2002. In certain aspects, the second model M2generated by the training 2300 is based on reducing a dimension of an input vector Xtof the input data set 2001.

[0054] In various aspects, the imputable data X'tis generated 2400 with the first model and the second model M2. More specifically, the imputable data X'tis generated 2400 with the first set of trained model parameters, the second set of trained model parameters, and the state vectors Stgenerated during the trainings 2200 and 2300. Further, the imputable data X'tis indexed to the second subset of time steps. The imputing module for generating 2400 the imputable data X'tcan further include an imputing algorithm similar in many respects to the imputing algorithm 1210 of the imputing module 1200 in FIG. 2. Thus, the imputing module can be configured to generate 2400 the imputable data X'tbased on a system of equations corresponding to a dimensionality of the input vectors Xt. The generated 2400 imputable data X'tcan be combined with the input data set 2001 as an input to a subsequent RNN to mitigate issues such as poor model performance and computational efficiency when training conventional RNNs with sparse data sets. Accordingly, the method 2000 can generate imputable data to supplement transaction data associated with a user to facilitate the development of predictive models for detecting fraudulent activity in any one of the payment devices linked to the user.

[0055] Further to the above, the method 2000 can include combining 2500 the input data set 2001 with the imputable data X't. In some aspects, the combining 2500 can include generating a new training data set Y with the imputable data X'tand portions of the input data set 2001 containing the sequential transaction data. Other configurations are contemplated by the present disclosure. For example, in some implementations, the method may include updating and / or appending data cells in the training data set associated with sparsities indexed to the second subset of time steps with the imputable data X'tas the imputable data is generated and / or after all of the imputable data has been generated.

[0056] In some aspects, the method 2000 further includes transmitting 2600 the imputable data X'tand portions of the input data set 2001 containing the sequential transaction data to a third neural network to develop a predictive model specific to the user’sspending behavior. In some aspects, the method 2000 is implemented in a continuous or semi-continuous manner such that a user’s validated transactions and transactions originating from a user’s payment device but denied by the user are incorporated into the models as they occur. For example, the method 2000 may be carried out on a modelling system which may be in communication with an API of a payment network server via an encrypted communication channel. In certain instances, the API of the payment network may communicate with a user device in an authenticated session to confirm or deny the validity of various transaction requests initiated on the user device, the results of which may then be communicated back to the modelling system for performing further updates and / or validations to the model.

[0057] FIG. 5 is a block diagram of a computer apparatus 3000 comprising data processing subsystems or components, according to at least one non-limiting aspect of the present disclosure. The subsystems shown in FIG. 6 are interconnected via a system bus 3010. Additional subsystems such as a printer 3018, keyboard 3026, fixed disk 3028 (or other memory comprising computer readable media), monitor 3022, which is coupled to a display adapter 3020, and others are shown. Peripherals and input / output (I / O) devices, which couple to an I / O controller 3012 (which can be a processor or other suitable controller), can be connected to the computer system by any number of means known in the art, such as a serial port 3024. For example, the serial port 3024 or external interface 3030 can be used to connect the computer apparatus to a wide area network such as the Internet, a mouse input device, or a scanner. The interconnection via system bus allows the central processor 3016 to communicate with each subsystem and to control the execution of instructions from system memory 3014 or the fixed disk 3028, as well as the exchange of information between subsystems. The system memory 3014 and / or the fixed disk 3028 may embody a computer readable medium.

[0058] FIG. 6 is a diagrammatic representation of an example computing system 4000 that includes a host machine 4002 within which a set of instructions to generate any one or more of the systems, models, and modules described herein and / or to perform any one or more of the methodologies described herein, according to at least one aspect of the present disclosure. In various aspects, the host machine 4002 operates as a standalone device or may be connected (e.g., networked) to other machines. In a networked deployment, the host machine 4002 may operate in the capacity of a server or a client machine in a server-client network environment, or as a peer machine in a peer-to-peer (or distributed) network environment. The host machine 4002 may be a computer or computing device, a personal computer (PC), a tablet PC, a set-top box (STB), a personal digital assistant (PDA), a cellular telephone, a portable music player (e.g., a portable hard drive audio device such asan Moving Picture Experts Group Audio Layer 3 (MP3) player), a web appliance, a network router, switch or bridge, or any machine capable of executing a set of instructions (sequential or otherwise) that specify actions to be taken by that machine. Further, while only a single machine is illustrated, the term “machine” shall also be taken to include any collection of machines that individually or jointly execute a set (or multiple sets) of instructions to perform any one or more of the methodologies discussed herein.

[0059] The example system 4000 includes the host machine 4002, running a host operating system (OS) 4004 on a processor or multiple processor(s) / processor core(s) 4006 (e.g., a central processing unit (CPU), a graphics processing unit (GPU), or both), and various memory nodes 4008. The host OS 4004 may include a hypervisor 4010 which is able to control the functions and / or communicate with a virtual machine (“VM”) 4012 running on machine readable media. The VM 4012 also may include a virtual CPU or vCPU 4014. The memory nodes 4008 may be linked or pinned to virtual memory nodes or vNodes 4016. When the memory node 4008 is linked or pinned to a corresponding vNode 4016, then data may be mapped directly from the memory nodes 4008 to the corresponding vNode 4016.

[0060] All the various components shown in host machine 4002 may be connected with and to each other, or communicate to each other via a bus (not shown) or via other coupling or communication channels or mechanisms. The host machine 4002 may further include a video display, audio device or other peripherals 4018 (e.g., a liquid crystal display (LCD), alpha-numeric input device(s) including, e.g., a keyboard, a cursor control device, e.g., a mouse, a voice recognition or biometric verification unit, an external drive, a signal generation device, e.g., a speaker,) a persistent storage device 4020 (also referred to as disk drive unit), and a network interface device 4022. The host machine 4002 may further include a data encryption module (not shown) to encrypt data. The components provided in the host machine 4002 are those typically found in computer systems that may be suitable for use with aspects of the present disclosure and are intended to represent a broad category of such computer components that are known in the art. Thus, the system 4000 can be a server, minicomputer, mainframe computer, or any other computer system. The computer may also include different bus configurations, networked platforms, multiprocessor platforms, and the like. Various operating systems may be used including UNIX, LINUX, WINDOWS, QNX ANDROID, IOS, CHROME, TIZEN, and other suitable operating systems.

[0061] The disk drive unit 4024 also may be a Solid-state Drive (SSD), a hard disk drive (HDD) or other includes a computer or machine-readable medium on which is stored one or more sets of instructions and data structures (e.g., data / instructions 4026) embodying or utilizing any one or more of the methodologies or functions described herein. Thedata / instructions 4026 also may reside, completely or at least partially, within the main memory node 4008 and / or within the processor(s) 4006 during execution thereof by the host machine 4002. The data / instructions 4026 may further be transmitted or received over a network 4028 via the network interface device 4022 utilizing any one of several well-known transfer protocols (e.g., Hyper Text Transfer Protocol (HTTP)).

[0062] The processor(s) 4006 and memory nodes 4008 also may comprise machine- readable media. The term "computer-readable medium" or “machine-readable medium” should be taken to include a single medium or multiple medium (e.g., a centralized or distributed database and / or associated caches and servers) that store the one or more sets of instructions. The term "computer-readable medium" shall also be taken to include any medium that is capable of storing, encoding, or carrying a set of instructions for execution by the host machine 4002 and that causes the host machine 4002 to perform any one or more of the methodologies of the present application, or that is capable of storing, encoding, or carrying data structures utilized by or associated with such a set of instructions. The term ’’computer-readable medium” shall accordingly be taken to include, but not be limited to, solid-state memories, optical and magnetic media, and carrier wave signals. Such media may also include, without limitation, hard disks, floppy disks, flash memory cards, digital video disks, random access memory (RAM), read only memory (ROM), and the like. The example aspects described herein may be implemented in an operating environment comprising software installed on a computer, in hardware, or in a combination of software and hardware.

[0063] One skilled in the art will recognize that Internet service may be configured to provide Internet access to one or more computing devices that are coupled to the Internet service, and that the computing devices may include one or more processors, buses, memory devices, display devices, input / output devices, and the like. Furthermore, those skilled in the art may appreciate that the Internet service may be coupled to one or more databases, repositories, servers, and the like, which may be utilized to implement any of the various aspects of the disclosure as described herein.

[0064] The computer program instructions also may be loaded onto a computer, a server, other programmable data processing apparatus, or other devices to cause a series of operational steps to be performed on the computer, other programmable apparatus or other devices to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide processes for implementing the functions / acts specified in the flowchart and / or block diagram block or blocks.

[0065] Suitable networks may include or interface with any one or more of, for instance, a local intranet, a PAN (Personal Area Network), a LAN (Local Area Network), a WAN (Wide Area Network), a MAN (Metropolitan Area Network), a virtual private network (VPN), a storage area network (SAN), a frame relay connection, an Advanced Intelligent Network (AIN) connection, a synchronous optical network (SONET) connection, a digital T1 , T3, E1 or E3 line, Digital Data Service (DDS) connection, DSL (Digital Subscriber Line) connection, an Ethernet connection, an ISDN (Integrated Services Digital Network) line, a dial-up port such as a V.90, V.34 or V.34bis analog modem connection, a cable modem, an ATM (Asynchronous Transfer Mode) connection, or an FDDI (Fiber Distributed Data Interface) or CDDI (Copper Distributed Data Interface) connection. Furthermore, communications may also include links to any of a variety of wireless networks, including WAP (Wireless Application Protocol), GPRS (General Packet Radio Service), GSM (Global System for Mobile Communication), CDMA (Code Division Multiple Access) or TDMA (Time Division Multiple Access), cellular phone networks, GPS (Global Positioning System), CDPD (cellular digital packet data), RIM (Research in Motion, Limited) duplex paging network, Bluetooth radio, or an IEEE 802.11-based radio frequency network. The network 4028 can further include or interface with any one or more of an RS-232 serial connection, an IEEE-1394 (Firewire) connection, a Fiber Channel connection, an IrDA (infrared) port, a SCSI (Small Computer Systems Interface) connection, a USB (Universal Serial Bus) connection or other wired or wireless, digital or analog interface or connection, mesh or Digi® networking.

[0066] In general, a cloud-based computing environment is a resource that typically combines the computational power of a large grouping of processors (such as within web servers) and / or that combines the storage capacity of a large grouping of computer memories or storage devices. Systems that provide cloud-based resources may be utilized exclusively by their owners or such systems may be accessible to outside users who deploy applications within the computing infrastructure to obtain the benefit of large computational or storage resources.

[0067] The cloud is formed, for example, by a network of web servers that comprise a plurality of computing devices, such as the host machine 4002, with each server 4030 (or at least a plurality thereof) providing processor and / or storage resources. These servers manage workloads provided by multiple users (e.g., cloud resource customers or other users). Typically, each user places workload demands upon the cloud that vary in real-time, sometimes dramatically. The nature and extent of these variations typically depends on the type of business associated with the user.

[0068] It is noteworthy that any hardware platform suitable for performing the processing described herein is suitable for use with the technology. The terms “computer-readablestorage medium” and “computer-readable storage media” as used herein refer to any medium or media that participate in providing instructions to a CPU for execution. Such media can take many forms, including, but not limited to, non-volatile media, volatile media, and transmission media. Non-volatile media include, for example, optical or magnetic disks, such as a fixed disk. Volatile media include dynamic memory, such as system RAM. Transmission media include coaxial cables, copper wire and fiber optics, among others, including the wires that comprise one aspect of a bus. Transmission media can also take the form of acoustic or light waves, such as those generated during radio frequency (RF) and infrared (IR) data communications. Common forms of computer-readable media include, for example, a flexible disk, a hard disk, magnetic tape, any other magnetic medium, a CD-ROM disk, digital video disk (DVD), any other optical medium, any other physical medium with patterns of marks or holes, a RAM, a PROM, an EPROM, an EEPROM, a FLASH EPROM, any other memory chip or data exchange adapter, a carrier wave, or any other medium from which a computer can read.

[0069] Various forms of computer-readable media may be involved in carrying one or more sequences of one or more instructions to a CPU for execution. A bus carries the data to system RAM, from which a CPU retrieves and executes the instructions. The instructions received by system RAM can optionally be stored on a fixed disk either before or after execution by a CPU.

[0070] Computer program code for carrying out operations for aspects of the present technology may be written in any combination of one or more programming languages, including an object oriented programming language such as Java, Smalltalk, C++, or the like and conventional procedural programming languages, such as the "C" programming language, Go, Python, or other programming languages, including assembly languages. The program code may execute entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer or entirely on the remote computer or server. In the latter scenario, the remote computer may be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or the connection may be made to an external computer (for example, through the Internet using an Internet Service Provider).

[0071] Examples of the devices, systems, and methods according to various aspects of the present disclosure are provided below in the following numbered clauses. An aspect of any of the devices(s), method(s) and / or system(s) may include any one or more than one, and any combination of, the numbered clauses described below.

[0072] Clause 1 - A computer-implemented method for modelling a sequence oftransactions with an imputed data set, the method comprising inputting, by a training module, an input data set into a first neural network. The input data set is indexed to a set of time steps, the set of time steps comprising a first subset of time steps and a second subset of time steps. The input data set comprises transaction data indexed to the first subset of time steps and sparsities indexed to the second subset of time steps. The transaction data is associated with the sequence of transactions. The method further comprises training, by the training module, the first neural network with the input data set to generate a first model based on a first set of trained model parameters and state vectors. The state vectors are produced by a transformation, based on the first model, of the input data set. The method further comprises training, by the training module, a second neural network with the input data set and the state vectors to generate a second model comprising a second set of trained model parameters. The second set of trained model parameters are based on a mapping of the input data set to the state vectors. The method further comprises generating, by a data imputing module, imputable data with the first set of trained model parameters, the second set of trained model parameters and the state vectors. The imputable data is indexed to the second subset of time steps.

[0073] Clause 2 - The computer-implemented method of clause 1 , wherein the first neural network is a Recurrent Neural Network.

[0074] Clause 3 - The computer-implemented method of clause 2, wherein the sparsities are representative of gaps within the sequence of transactions.

[0075] Clause 4 - The computer-implemented method of clause 3, wherein the first neural network is based on:(ti—ti-i)Mi(Sti-1,Xt.) wherein: tj is a member of the first subset of time steps; tj_xis a time step preceding tj in the first subset;St. is a state vector corresponding to t;;St._iis a state vector corresponding to;Xt)is an input vector of the input data set corresponding to tj; andMi comprises the first set of trained model parameters, wherein Mi is function of St._iand Xt..

[0076] Clause 5 - The computer-implemented method of any one of clauses 1-4, wherein the imputable data is generated based on a system of equations.

[0077] Clause 6 - The computer-implemented method of clause 5, wherein the input data set comprises input vectors, the input vectors comprising the transaction data.

[0078] Clause 7 - The computer-implemented method of clause 6, wherein the system of equations comprises a number of equations corresponding to a dimension of the input vectors.

[0079] Clause 8 - The computer-implemented method of clause 7, wherein the system of equations consists of a number of equations equal to the dimension of the input vectors.

[0080] Clause 9 - The computer-implemented method of any one of clauses 1-4, further comprising generating, by the data imputing module, a set of imputable data, wherein the set of imputable data is generated with a set of systems of equations based on the first set of trained model parameters, the second set of trained model parameters and the state vectors.

[0081] Clause 10 - The computer-implemented method of any one of clauses 1-9, the method further comprising transmitting an imputed data set to a third neural network. The imputed data set comprises the transaction data indexed to the first subset of time steps and the imputable data indexed to the second subset of time steps.

[0082] Clause 11 - The computer-implemented method of clause 10, the method further comprising training the third neural network to generate a third set of trained model parameters for detecting fraudulent behavior.

[0083] Clause 12 - The computer-implemented method of any one of clauses 1-12, wherein the transaction data comprises a sequence of transactions associated with multiple payment devices.

[0084] Clause 13 - The computer-implemented method of clause 12, wherein the multiple payment devices are associated with a single entity comprising one or more users.

[0085] Clause 14 - The computer-implemented method of clause 13, wherein the multiple payment devices are associated with a single user.

[0086] Clause 15 - The computer-implemented method of any one of clauses 1-14, wherein the second neural network is a Fully Connected Neural Network.

[0087] Clause 16 -The computer-implemented method of any one of clauses 1-15, wherein the transactions of the sequence of transactions occur on an irregular basis.

[0088] Clause 17 - The computer-implemented method of clause 16, wherein the sparsities are unevenly distributed throughout the input data set.

[0089] Clause 18 - The computer-implemented method of clause 17, wherein the transaction data are unevenly distributed among the sparsities.

[0090] Clause 19 - A system for generating an imputed data set from transaction data, the system comprising a training module for modelling the transaction data. The training model comprises a first training model and a second training model. The first training model comprises a Recurrent Neural Network (RNN) configured to accommodate sparsities in the transaction data. The first training model is configured to output state vectors and first model parameters based on a training with the transaction data. The second training model comprises a fully connected neural network configured to output second model parameters based on a mapping of the transaction data to the state vectors. The system further comprises a data imputing module for generating new data to satisfy the sparsities. The new data is based on the transaction data, the state vectors, the first model parameters, and the second model parameters.

[0091] Clause 20 - The system of clause 19, wherein the sparsities are representative of gaps in the transaction data.

[0092] Further, it is understood that any one or more of the following-described forms, expressions of forms, examples, can be combined with any one or more of the other following-described forms, expressions of forms, and examples.

[0093] While several forms have been illustrated and described, it is not the intention of Applicant to restrict or limit the scope of the appended claims to such detail. Numerous modifications, variations, changes, substitutions, combinations, and equivalents to those forms may be implemented and will occur to those skilled in the art without departing from the scope of the present disclosure. Moreover, the structure of each element associated with the described forms can be alternatively described as a means for providing the function performed by the element. Also, where materials are disclosed for certain components, other materials may be used. It is therefore to be understood that the foregoing description and the appended claims are intended to cover all such modifications, combinations, and variations as falling within the scope of the disclosed forms. The appended claims are intended to cover all such modifications, variations, changes, substitutions, modifications, andequivalents.

[0094] As used herein, a “server” may include one or more computing devices which can be individual, stand-alone machines located at the same or different locations, may be owned or operated by the same or different entities, and may further be one or more clusters of distributed computers or “virtual” machines housed within a datacenter. It should be understood and appreciated by a person of skill in the art that functions performed by one “server” can be spread across multiple disparate computing devices for various reasons. As used herein, a “server” is intended to refer to all such scenarios and should not be construed or limited to one specific configuration. Further, a server as described herein may, but need not, reside at (or be operated by) a merchant, a payment network, a financial institution, a healthcare provider, a social media provider, a government agency, or agents of any of the aforementioned entities. The term “server” may also refer to or include one or more processors or computers, storage devices, or similar computer arrangements that are operated by or facilitate communication and processing for multiple parties in a network environment, such as the Internet, although it will be appreciated that communication may be facilitated over one or more public or private network environments and that various other arrangements are possible. Further, multiple computers, e.g., servers, or other computerized devices, e.g., point-of-sale devices, directly or indirectly communicating in the network environment may constitute a “system,” such as a merchant's point-of-sale system. Reference to “a server” or “a processor,” as used herein, may refer to a previously recited server and / or processor that is recited as performing a previous step or function, a different server and / or processor, and / or a combination of servers and / or processors. For example, as used in the specification and the claims, a first server and / or a first processor that is recited as performing a first step or function may refer to the same or different server and / or a processor recited as performing a second step or function.

[0095] The term “system” may refer to one or more computing devices or combinations of computing devices (e.g., processors, servers, client devices, software applications, modules, components of such, and / or the like). For example, a system may include a plurality of computing devices that include software applications, where the plurality of computing devices are connected via a network.

[0096] As used herein, a “server computer” may describe a powerful computer or cluster of computers. For example, the server computer can be a large mainframe, a minicomputer cluster, or a group of servers functioning as a unit. The server computer may be associated with an entity such as a payment processing network, a wallet provider, a merchant, an authentication cloud, an acquirer or an issuer. In one example, the server computer may be a database server coupled to a Web server. The server computer may be coupled to adatabase and may include any hardware, software, other logic, or combination of the preceding for servicing the requests from one or more client computers. The server computer may comprise one or more computational apparatuses and may use any of a variety of computing structures, arrangements, and compilations for servicing the requests from one or more client computers. In some embodiments or aspects, the server computer may provide and / or support payment network cloud service.

[0097] Reference to “a device,” “a server,” “a processor,” and / or the like, as used herein, may refer to a previously recited device, server, or processor that is recited as performing a previous step or function, a different server or processor, and / or a combination of servers and / or processors. For example, as used in the specification and the claims, a first server or a first processor that is recited as performing a first step or a first function may refer to the same or different server or the same or different processor recited as performing a second step or a second function.

[0098] One or more components may be referred to herein as “configured to,” “configurable to,” “operable / operative to,” “adapted / adaptable,” “able to,” “conformable / conformed to,” etc. Those skilled in the art will recognize that “configured to” can generally encompass active-state components and / or inactive-state components and / or standby-state components, unless context requires otherwise.

[0099] Those skilled in the art will recognize that, in general, terms used herein, and especially in the appended claims (e.g., bodies of the appended claims) are generally intended as “open” terms (e.g., the term “including” should be interpreted as “including but not limited to,” the term “having” should be interpreted as “having at least,” the term “includes” should be interpreted as “includes but is not limited to,” etc.). It will be further understood by those within the art that if a specific number of an introduced claim recitation is intended, such an intent will be explicitly recited in the claim, and in the absence of such recitation no such intent is present. For example, as an aid to understanding, the following appended claims may contain usage of the introductory phrases “at least one” and “one or more” to introduce claim recitations. However, the use of such phrases should not be construed to imply that the introduction of a claim recitation by the indefinite articles “a” or “an” limits any particular claim containing such introduced claim recitation to claims containing only one such recitation, even when the same claim includes the introductory phrases “one or more” or “at least one” and indefinite articles such as “a” or “an” (e.g., “a” and / or “an” should typically be interpreted to mean “at least one” or “one or more”); the same holds true for the use of definite articles used to introduce claim recitations.

[0100] The term “substantially”, “about”, or “approximately” as used in the presentdisclosure, unless otherwise specified, means an acceptable error for a particular value as determined by one of ordinary skill in the art, which depends in part on how the value is measured or determined. In certain aspects, the term “substantially”, “about”, or “approximately” means within 1 , 2, 3, or 4 standard deviations. In certain aspects, the term “substantially”, “about”, or “approximately” means within 50%, 20%, 15%, 10%, 9%, 8%, 7%, 6%, 5%, 4%, 3%, 2%, 1%, 0.5%, or 0.05% of a given value or range.

[0101] In addition, even if a specific number of an introduced claim recitation is explicitly recited, those skilled in the art will recognize that such recitation should typically be interpreted to mean at least the recited number (e.g., the bare recitation of “two recitations,” without other modifiers, typically means at least two recitations, or two or more recitations). Furthermore, in those instances where a convention analogous to “at least one of A, B, and C, etc.” is used, in general such a construction is intended in the sense one having skill in the art would understand the convention (e.g., “a system having at least one of A, B, and C” would include but not be limited to systems that have A alone, B alone, C alone, A and B together, A and C together, B and C together, and / or A, B, and C together, etc.). In those instances where a convention analogous to “at least one of A, B, or C, etc.” is used, in general such a construction is intended in the sense one having skill in the art would understand the convention (e.g., “a system having at least one of A, B, or C” would include but not be limited to systems that have A alone, B alone, C alone, A and B together, A and C together, B and C together, and / or A, B, and C together, etc.). It will be further understood by those within the art that typically a disjunctive word and / or phrase presenting two or more alternative terms, whether in the description, claims, or drawings, should be understood to contemplate the possibilities of including one of the terms, either of the terms, or both terms unless context dictates otherwise. For example, the phrase “A or B” will be typically understood to include the possibilities of “A” or “B” or “A and B.”

[0102] With respect to the appended claims, those skilled in the art will appreciate that recited operations therein may generally be performed in any order. Also, although various operational flow diagrams are presented in a sequence(s), it should be understood that the various operations may be performed in other orders than those which are illustrated, or may be performed concurrently. Examples of such alternate orderings may include overlapping, interleaved, interrupted, reordered, incremental, preparatory, supplemental, simultaneous, reverse, or other variant orderings, unless context dictates otherwise. Furthermore, terms like “responsive to,” “related to,” or other past-tense adjectives are generally not intended to exclude such variants, unless context dictates otherwise.

[0103] It is worthy to note that any reference to “one aspect,” “an aspect,” “an exemplification,” “one exemplification,” and the like means that a particular feature, structure,or characteristic described in connection with the aspect is included in at least one aspect. Thus, appearances of the phrases “in one aspect,” “in an aspect,” “in an exemplification,” and “in one exemplification” in various places throughout the specification are not necessarily all referring to the same aspect. Furthermore, the particular features, structures or characteristics may be combined in any suitable manner in one or more aspects.

[0104] As used herein, the singular form of “a”, “an”, and “the” include the plural references unless the context clearly dictates otherwise.

[0105] Any patent application, patent, non-patent publication, or other disclosure material referred to in this specification and / or listed in any Application Data Sheet is incorporated by reference herein, to the extent that the incorporated materials is not inconsistent herewith. As such, and to the extent necessary, the disclosure as explicitly set forth herein supersedes any conflicting material incorporated herein by reference. Any material, or portion thereof, that is said to be incorporated by reference herein, but which conflicts with existing definitions, statements, or other disclosure material set forth herein will only be incorporated to the extent that no conflict arises between that incorporated material and the existing disclosure material.

[0106] In summary, numerous benefits have been described which result from employing the concepts described herein. The foregoing description of the one or more forms has been presented for purposes of illustration and description. It is not intended to be exhaustive or limiting to the precise form disclosed. Modifications or variations are possible in light of the above teachings. The one or more forms were chosen and described in order to illustrate principles and practical application to thereby enable one of ordinary skill in the art to utilize the various forms and with various modifications as are suited to the particular use contemplated. It is intended that the claims submitted herewith define the overall scope.

Claims

CLAIMSWhat is claimed is:

1. A computer-implemented method for modelling a sequence of transactions with an imputed data set, the method comprising: inputting, by a training module, an input data set into a first neural network, wherein the input data set is indexed to a set of time steps, the set of time steps comprising a first subset of time steps and a second subset of time steps, the input data set comprising: transaction data indexed to the first subset of time steps, wherein the transaction data is associated with the sequence of transactions; and sparsities indexed to the second subset of time steps; training, by the training module, the first neural network with the input data set, to generate: a first model based on a first set of trained model parameters; and state vectors, wherein the state vectors are produced by a transformation, based on the first model, of the input data set; training, by the training module, a second neural network with the input data set and the state vectors, to generate a second model comprising a second set of trained model parameters, wherein the second set of trained model parameters are based on a mapping of the input data set to the state vectors; and generating, by a data imputing module, imputable data with the first set of trained model parameters, the second set of trained model parameters and the state vectors, wherein the imputable data is indexed to the second subset of time steps.

2. The computer-implemented method of claim 1, wherein the first neural network is a Recurrent Neural Network (RNN).

3. The computer-implemented method of claim 2, wherein the sparsities are representative of gaps within the sequence of transactions.

4. The computer-implemented method of claim 3, wherein the first neural network is based on:(ti ti-i)Mi(Sti-1,Xt.) wherein: tj is a member of the first subset of time steps; tj_xis a time step preceding tj in the first subset;St. is a state vector corresponding to t;;St._1is a state vector corresponding to tj-i ;Xt. is an input vector of the input data set corresponding to tj; andMi comprises the first set of trained model parameters, wherein Mi is function of St._iand Xt..

5. The computer-implemented method of claim 3, wherein the imputable data is generated based on a system of equations.

6. The computer-implemented method of claim 5, wherein the input data set comprises input vectors, the input vectors comprising the transaction data.

7. The computer-implemented method of claim 6, wherein the system of equations comprises a number of equations corresponding to a dimension of the input vectors.

8. The computer-implemented method of claim 7, wherein the system of equations consists of a number of equations equal to the dimension of the input vectors.

9. The computer-implemented method of claim 1, further comprising generating, by the data imputing module, a set of imputable data, wherein the set of imputable data is generated with a set of systems of equations based on the first set of trained model parameters, the second set of trained model parameters and the state vectors.

10. The computer-implemented method of claim 1, the method further comprising transmitting an imputed data set to a third neural network, the imputed data set comprising: the transaction data indexed to the first subset of time steps; and the imputable data indexed to the second subset of time steps.11 . The computer-implemented method of claim 10, the method further comprisingtraining the third neural network to generate a third set of trained model parameters for detecting fraudulent behavior.

12. The computer-implemented method of claim 1, wherein the transaction data comprises a sequence of transactions associated with multiple payment devices.

13. The computer-implemented method of claim 12, wherein the multiple payment devices are associated with a single entity comprising one or more users.

14. The computer-implemented method of claim 13, wherein the multiple payment devices are associated with a single user.

15. The computer-implemented method of claim 1 , wherein the second neural network is a Fully Connected Neural Network (FCNN).

16. The computer-implemented method of claim 1, wherein the transactions of the sequence of transactions occur on an irregular basis.

17. The computer-implemented method of claim 16, wherein the sparsities are unevenly distributed throughout the input data set.

18. The computer-implemented method of claim 17, wherein the transaction data are unevenly distributed among the sparsities.

19. A system for generating an imputed data set from transaction data, the system comprising: a training module for modelling the transaction data, the training module comprising: a first training model comprising a Recurrent Neural Network (RNN), wherein the RNN is configured to accommodate sparsities in the transaction data, wherein the first training model is configured to output state vectors and first model parameters based on a training with the transaction data; and a second training model comprising a fully connected neural network configured to output second model parameters based on a mapping of the transaction data to the state vectors; and a data imputing module for generating new data to satisfy the sparsities, wherein the new data is based on the transaction data, the state vectors, the first model parameters, and the second model parameters.

20. The system of claim 19, wherein the sparsities are representative of gaps in the transaction data.

Images