Method, system, and computer program product for time series analysis using a time interval embedding based attention mechanism

Abstract

Methods, systems, and computer program products for time series analysis using a time interval embedding based attention mechanism are provided. An example method may include receiving training data associated with a time series of a plurality of events, determining a plurality of time intervals between each pair of events of the plurality of events of the time series, generating a time interval embedding for each time interval of the plurality of time intervals to provide a plurality of time interval embeddings, calculating a plurality of attention weights for an attention mechanism of a machine learning model based on the plurality of time interval embeddings, and generating a trained machine learning model based on the attention weights for the attention mechanism of the machine learning model.

Description

METHOD, SYSTEM, AND COMPUTER PROGRAM PRODUCT FOR TIME SERIES ANALYSIS USING A TIME INTERVAL EMBEDDING BASED ATTENTION MECHANISMCROSS REFERENCE TO RELATED APPLICATION

[0001] This application claims priority to United States Provisional Patent Application No. 63 / 636,145, filed on April 19, 2024, the disclosure of which is hereby incorporated by reference in its entirety.BACKGROUND1 . Technical Field

[0002] This disclosure relates generally to time series analysis and, in some nonlimiting embodiments or aspects, to methods, systems, and computer program products for time series analysis using a time interval embedding based attention mechanism.2. Technical Considerations

[0003] A time series may refer to a series of data points provided (e.g., indexed, listed, graphed, etc.) in time order. In some instances, a time series may include a sequence of data points taken at successive, equally spaced points in time. In such an instance, the time series may be a sequence of discrete-time data. Machine learning models may be used to analyze time series data and make predictions about future values.

[0004] However, machine learning models that may be used for analysis of time series data may include an attention mechanism that is designed for time series data that is evenly spaced. With this, such machine learning models may not be able to account for an amount of time in a time interval between events in the time series. Accordingly, some machine learning models may not be capable of accurately performing analysis on time series data that has unevenly spaced time intervals between events in the time series.SUMMARY

[0005] Accordingly, provided are improved methods, systems, and computer program products for time series analysis using a time interval embedding based attention mechanism.

[0006] According to non-limiting embodiments or aspects, provided is a computer implemented method for time series analysis using a time interval embedding based attention mechanism, including: receiving, with at least one processor, training data associated with a time series of a plurality of events, determining, with at least one processor, a plurality of time intervals between each pair of events of the plurality of events of the time series, generating, with at least one processor, a time interval embedding for each time interval of the plurality of time intervals to provide a plurality of time interval embeddings, calculating, with at least one processor, a plurality of attention weights for an attention mechanism of a machine learning model based on the plurality of time interval embeddings, and generating, with at least one processor, a trained machine learning model based on the attention weights for the attention mechanism of the machine learning model.

[0007] According to non-limiting embodiments or aspects, provided is a system for time series analysis using a time interval embedding based attention mechanism including, at least one processor configured to receive data associated with a time series of a plurality of events, determine a plurality of time intervals between each pair of events of the plurality of events of the time series; generate a time interval embedding for each time interval of the plurality of time intervals to provide a plurality of time interval embeddings, calculate a plurality of attention weights for an attention mechanism of a machine learning model based on the plurality of time interval embeddings, and generate a trained machine learning model based on the attention weights for the attention mechanism of the machine learning model.

[0008] According to non-limiting embodiments or aspects, provided is a computer program product for time series analysis using a time interval embedding based attention mechanism, including at least one non-transitory computer-readable medium including program instructions that, when executed by at least one processor, cause the at least one processor to receive training data associated with a time series of a plurality of events, determine a plurality of time intervals between each pair of events of the plurality of events of the time series, generate a time interval embedding for each time interval of the plurality of time intervals to provide a plurality of time intervalembeddings, calculate a plurality of attention weights for an attention mechanism of a machine learning model based on the plurality of time interval embeddings, and generate a trained machine learning model based on the attention weights for the attention mechanism of the machine learning model.

[0009] Further non-limiting embodiments or aspects will be set forth in the following numbered clauses:

[0010] Clause 1 : A computer-implemented method, comprising: receiving, with at least one processor, training data associated with a time series of a plurality of events; determining, with at least one processor, a plurality of time intervals between each pair of events of the plurality of events of the time series; generating, with at least one processor, a time interval embedding for each time interval of the plurality of time intervals to provide a plurality of time interval embeddings; calculating, with at least one processor, a plurality of attention weights for an attention mechanism of a machine learning model based on the plurality of time interval embeddings; and generating, with at least one processor, a trained machine learning model based on the attention weights for the attention mechanism of the machine learning model.

[0011] Clause 2: The computer-implemented method of clause 1 , wherein generating the time interval embedding for each time interval of the plurality of time intervals to provide the plurality of time interval embeddings comprises: determining an amount of time involved in each time interval of the plurality of time intervals; determining a discrete time interval of a plurality of discrete time intervals; and generating the time interval embedding for each time interval of the plurality of time intervals based on the discrete time interval associated with each time interval of the plurality of time intervals.

[0012] Clause 3: The computer-implemented method of clause 1 or 2, further comprising: discretizing each time interval of the plurality of time intervals based on the amount of time involved in each time interval to provide the plurality of discrete time intervals.

[0013] Clause 4: The computer-implemented method of any of clauses 1 -3, further comprising: determining whether to perform an action based on an inference of the trained machine learning model.

[0014] Clause 5: The computer-implemented method of any of clauses 1 -4, further comprising: calculating a plurality of attention scores of the machine learning model based on the plurality of time interval embeddings and the plurality of events, whereineach attention score comprises an output of the attention mechanism of the machine learning model based on an input provided to the attention mechanism; and wherein calculating the plurality of attention weights for the attention mechanism of the machine learning model comprises: calculating the plurality of attention weights for the attention mechanism of the machine learning model based on the plurality of attention scores of the machine learning model.

[0015] Clause 6: The computer-implemented method of any of clauses 1 -5, wherein calculating the plurality of attention weights for the attention mechanism of the machine learning model comprises: executing a Tanh alignment function based on the plurality of time interval embeddings and the plurality of events to provide a time- embedded alignment factor; determining a first plurality of attention weights for the attention mechanism based on the plurality of time interval embeddings and the plurality of events; and multiplying the first plurality of attention weights for the attention mechanism by the time-embedded alignment factor to provide the plurality of attention weights for the attention mechanism of the machine learning model.

[0016] Clause 7: The computer-implemented method of any of clauses 1 -6, wherein the machine learning model comprises a recurrent neural network (RNN) machine learning model.

[0017] Clause 8: A system for time series analysis using a time interval embedding based attention mechanism, comprising: at least one processor configured to: receive data associated with a time series of a plurality of events; determine a plurality of time intervals between each pair of events of the plurality of events of the time series; generate a time interval embedding for each time interval of the plurality of time intervals to provide a plurality of time interval embeddings; calculate a plurality of attention weights for an attention mechanism of a machine learning model based on the plurality of time interval embeddings; and generate a trained machine learning model based on the attention weights for the attention mechanism of the machine learning model.

[0018] Clause 9: The system of clause 8, wherein, when generating the time interval embedding for each time interval of the plurality of time intervals to provide the plurality of time interval embeddings, the at least one processor is configured to: determine an amount of time involved in each time interval of the plurality of time intervals; determine a discrete time interval of a plurality of discrete time intervals; and generate the timeinterval embedding for each time interval of the plurality of time intervals based on the discrete time interval associated with each time interval of the plurality of time intervals.

[0019] Clause 10: The system of clause 8 or 9, wherein the at least one processor is further configured to: discretize each time interval of the plurality of time intervals based on the amount of time involved in each time interval to provide the plurality of discrete time intervals.

[0020] Clause 1 1 : The system of any of clauses 8-10, wherein the at least one processor is further configured to: determine whether to perform an action based on an inference of the trained machine learning model.

[0021] Clause 12: The system of any of clauses 8-1 1 , wherein the at least one processor is further configured to: calculate a plurality of attention scores of the machine learning model based on the plurality of time interval embeddings and the plurality of events, wherein each attention score comprises an output of the attention mechanism of the machine learning model based on an input provided to the attention mechanism; and wherein, when calculating the plurality of attention weights for the attention mechanism of the machine learning model, the at least one processor is configured to: calculate the plurality of attention weights for the attention mechanism of the machine learning model based on the plurality of attention scores of the machine learning model.

[0022] Clause 13: The system of any of clauses 8-12, wherein, when calculating the plurality of attention weights for the attention mechanism of the machine learning model, the at least one processor is configured to: execute a Tanh alignment function based on the plurality of time interval embeddings and the plurality of events to provide a time-embedded alignment factor; determine a first plurality of attention weights for the attention mechanism based on the plurality of time interval embeddings and the plurality of events; and multiply the first plurality of attention weights for the attention mechanism by the time-embedded alignment factor to provide the plurality of attention weights for the attention mechanism of the machine learning model.

[0023] Clause 14: The system of any of clauses 8-13, wherein the machine learning model comprises a recurrent neural network (RNN) machine learning model.

[0024] Clause 15: A computer program product, including at least one non- transitory computer-readable medium including program instructions that, when executed by at least one processor, cause the at least one processor to: receive training data associated with a time series of a plurality of events; determine a pluralityof time intervals between each pair of events of the plurality of events of the time series; generate a time interval embedding for each time interval of the plurality of time intervals to provide a plurality of time interval embeddings; calculate a plurality of attention weights for an attention mechanism of a machine learning model based on the plurality of time interval embeddings; and generate a trained machine learning model based on the attention weights for the attention mechanism of the machine learning model.

[0025] Clause 16: The computer program product of clause 15, wherein, the one or more program instructions that cause the at least one processor to generate the time interval embedding for each time interval of the plurality of time intervals to provide the plurality of time interval embeddings, cause the at least one processor to: determine an amount of time involved in each time interval of the plurality of time intervals; determine a discrete time interval of a plurality of discrete time intervals; and generate the time interval embedding for each time interval of the plurality of time intervals based on the discrete time interval associated with each time interval of the plurality of time intervals.

[0026] Clause 17: The computer program product of clause 15 or 16, wherein the one or more program instructions further cause the at least one processor to: discretize each time interval of the plurality of time intervals based on the amount of time involved in each time interval to provide the plurality of discrete time intervals.

[0027] Clause 18: The computer program product of any of clauses 15-17, wherein the one or more program instructions further cause the at least one processor to: determine whether to perform an action based on an inference of the trained machine learning model.

[0028] Clause 19: The computer program product of any of clauses 15-18, wherein the one or more program instructions further cause the at least one processor to: calculate a plurality of attention scores of the machine learning model based on the plurality of time interval embeddings and the plurality of events, wherein each attention score comprises an output of the attention mechanism of the machine learning model based on an input provided to the attention mechanism; and wherein, the one or more program instructions that cause the at least one processor to calculate the plurality of attention weights for the attention mechanism of the machine learning model, cause the at least one processor to: calculate the plurality of attention weights for theattention mechanism of the machine learning model based on the plurality of attention scores of the machine learning model.

[0029] Clause 20: The computer program product of any of clauses 15-19, wherein, the one or more program instructions that cause the at least one processor to calculate the plurality of attention weights for the attention mechanism of the machine learning model, cause the at least one processor to: execute a Tanh alignment function based on the plurality of time interval embeddings and the plurality of events to provide a time-embedded alignment factor; determine a first plurality of attention weights for the attention mechanism based on the plurality of time interval embeddings and the plurality of events; and multiply the first plurality of attention weights for the attention mechanism by the time-embedded alignment factor to provide the plurality of attention weights for the attention mechanism of the machine learning model.

[0030] Clause 21 : The computer program product of any of clauses 15-20, wherein the machine learning model comprises a recurrent neural network (RNN) machine learning model.

[0031] These and other features and characteristics of the present disclosure, as well as the methods of operation and functions of the related elements of structures and the combination of parts and economies of manufacture, will become more apparent upon consideration of the following description and the appended claims with reference to the accompanying drawings, all of which form a part of this specification, wherein like reference numerals designate corresponding parts in the various figures. It is to be expressly understood, however, that the drawings are for the purpose of illustration and description only and are not intended as a definition of the limits of the disclosed subject matter.BRIEF DESCRIPTION OF THE DRAWINGS

[0032] Additional advantages and details are explained in greater detail below with reference to the non-limiting, exemplary embodiments that are illustrated in the accompanying schematic figures, in which:

[0033] FIG. 1 is a schematic diagram of a system for time series analysis using a time interval embedding based attention mechanism, according to some non-limiting embodiments or aspects;

[0034] FIG. 2 is a flow diagram of a process for time series analysis using a time interval embedding based attention mechanism, according to some non-limiting embodiments or aspects;

[0035] FIGS. 3A-3D are schematic diagrams of an exemplary implementation of a system and / or method for time series analysis using a time interval embedding based attention mechanism, according to some non-limiting embodiments or aspects;

[0036] FIG. 4 is a diagram of an exemplary environment in which systems, methods, and / or computer program products, described herein, may be implemented, according to some non-limiting embodiments or aspects; and

[0037] FIG. 5 is a schematic diagram of example components of one or more devices of FIG. 1 and / or FIG. 4, according to some non-limiting embodiments or aspects.DETAILED DESCRIPTION

[0038] For purposes of the description hereinafter, the terms “end,” “upper,” “lower,” “right,” “left,” “vertical,” “horizontal,” “top,” “bottom,” “lateral,” “longitudinal,” and derivatives thereof shall relate to the embodiments as they are oriented in the drawing figures. However, it is to be understood that the present disclosure may assume various alternative variations and step sequences, except where expressly specified to the contrary. It is also to be understood that the specific devices and processes illustrated in the attached drawings, and described in the following specification, are simply exemplary and non-limiting embodiments or aspects of the disclosed subject matter. Hence, specific dimensions and other physical characteristics related to the embodiments or aspects disclosed herein are not to be considered as limiting.

[0039] Some non-limiting embodiments or aspects are described herein in connection with thresholds. As used herein, satisfying a threshold may refer to a value being greater than the threshold, more than the threshold, higher than the threshold, greater than or equal to the threshold, less than the threshold, fewer than the threshold, lower than the threshold, less than or equal to the threshold, equal to the threshold, etc.

[0040] No aspect, component, element, structure, act, step, function, instruction, and / or the like used herein should be construed as critical or essential unless explicitly described as such. Also, as used herein, the articles “a” and “an” are intended to include one or more items and may be used interchangeably with “one or more” and“at least one.” Furthermore, as used herein, the term “set” is intended to include one or more items (e.g., related items, unrelated items, a combination of related and unrelated items, and / or the like) and may be used interchangeably with “one or more” or “at least one.” Where only one item is intended, the term “one” or similar language is used. Also, as used herein, the terms “has,” “have,” “having,” or the like are intended to be open-ended terms. Further, the phrase “based on” is intended to mean “based at least partially on” unless explicitly stated otherwise. In addition, reference to an action being “based on” a condition may refer to the action being “in response to” the condition. For example, the phrases “based on” and “in response to” may, in some non-limiting embodiments or aspects, refer to a condition for automatically triggering an action (e.g., a specific operation of an electronic device, such as a computing device, a processor, and / or the like).

[0041] As used herein, the term “acquirer institution” may refer to an entity licensed and / or approved by a transaction service provider to originate transactions (e.g., payment transactions) using a payment device associated with the transaction service provider. The transactions the acquirer institution may originate may include payment transactions (e.g., purchases, original credit transactions (OCTs), account funding transactions (AFTs), and / or the like). In some non-limiting embodiments or aspects, an acquirer institution may be a financial institution, such as a bank. As used herein, the term “acquirer system” may refer to one or more computing devices operated by or on behalf of an acquirer institution, such as a server computer executing one or more software applications.

[0042] As used herein, the term “account identifier” may include one or more primary account numbers (PANs), tokens, or other identifiers associated with a customer account. The term “token” may refer to an identifier that is used as a substitute or replacement identifier for an original account identifier, such as a PAN. Account identifiers may be alphanumeric or any combination of characters and / or symbols. Tokens may be associated with a PAN or other original account identifier in one or more data structures (e.g., one or more databases, and / or the like) such that they may be used to conduct a transaction without directly using the original account identifier. In some examples, an original account identifier, such as a PAN, may be associated with a plurality of tokens for different individuals or purposes.

[0043] As used herein, the term “communication” may refer to the reception, receipt, transmission, transfer, provision, and / or the like of data (e.g., information, signals,messages, instructions, commands, and / or the like). For one unit (e.g., a device, a system, a component of a device or system, combinations thereof, and / or the like) to be in communication with another unit means that the one unit is able to directly or indirectly receive information from and / or transmit information to the other unit. This may refer to a direct or indirect connection (e.g., a direct communication connection, an indirect communication connection, and / or the like) that is wired and / or wireless in nature. Additionally, two units may be in communication with each other even though the information transmitted may be modified, processed, relayed, and / or routed between the first and second unit. For example, a first unit may be in communication with a second unit even though the first unit passively receives information and does not actively transmit information to the second unit. As another example, a first unit may be in communication with a second unit if at least one intermediary unit processes information received from the first unit and communicates the processed information to the second unit. In some non-limiting embodiments or aspects, a message may refer to a network packet (e.g., a data packet and / or the like) that includes data. It will be appreciated that numerous other arrangements are possible.

[0044] As used herein, the term “computing device” may refer to one or more electronic devices configured to process data. A computing device may, in some examples, include the necessary components to receive, process, and output data, such as a processor, a display, a memory, an input device, a network interface, and / or the like. A computing device may be a mobile device. As an example, a mobile device may include a cellular phone (e.g., a smartphone or standard cellular phone), a portable computer, a wearable device (e.g., watches, glasses, lenses, clothing, and / or the like), a personal digital assistant (PDA), and / or other like devices. A computing device may also be a desktop computer or other form of non-mobile computer.

[0045] As used herein, the term “server” may refer to or include one or more computing devices that are operated by or facilitate communication and processing for multiple parties in a network environment, such as the Internet, although it will be appreciated that communication may be facilitated over one or more public or private network environments and that various other arrangements are possible. Further, multiple computing devices (e.g., servers, point-of-sale (POS) devices, mobile devices, etc.) directly or indirectly communicating in the network environment may constitute a “system.”

[0046] As used herein, the term “system” may refer to one or more computing devices or combinations of computing devices and / or components of such (e.g., processors, servers, client devices, software applications, and / or the like). Reference to “a device,” “a server,” “a processor,” and / or the like, as used herein, may refer to a previously-recited device, server, or processor that is recited as performing a previous step or function, a different device, server, or processor, and / or a combination of devices, servers, and / or processors. For example, as used in the specification and the claims, a first device, a first server, or a first processor that is recited as performing a first step or a first function may refer to the same or different device, server, or processor recited as performing a second step or a second function.

[0047] As used herein, the term “issuer institution” may refer to one or more entities, such as a bank, that provide accounts to customers for conducting transactions (e.g., payment transactions), such as initiating credit and / or debit payments. For example, an issuer institution may provide an account identifier, such as a PAN, to a customer that uniquely identifies one or more accounts associated with that customer. The account identifier may be embodied on a portable financial device, such as a physical financial instrument, e.g., a payment card, and / or may be electronic and used for electronic payments. The term “issuer system” refers to one or more computer devices operated by or on behalf of an issuer institution, such as a server computer executing one or more software applications. For example, an issuer system may include one or more authorization servers for authorizing a transaction.

[0048] As used herein, the term “merchant” may refer to an individual or entity that provides goods and / or services, or access to goods and / or services, to customers based on a transaction, such as a payment transaction. The term “merchant” or “merchant system” may also refer to one or more computer systems operated by or on behalf of a merchant, such as a server computer executing one or more software applications.

[0049] As used herein, the term “payment device” may refer to an electronic payment device, a portable financial device (e.g., a payment card, such as a credit or debit card), a gift card, a smartcard, smart media, a payroll card, a healthcare card, a wristband, a machine-readable medium containing account information, a keychain device or fob, a radio frequency identification (RFID) transponder, a retailer discount or loyalty card, a cellular phone, an electronic wallet mobile application, a PDA, a pager, a security card, a computing device, an access card, a wireless terminal, atransponder, and / or the like. In some non-limiting embodiments or aspects, the payment device may include volatile or non-volatile memory to store information (e.g., an account identifier, a name of the account holder, and / or the like).

[0050] As used herein, a “point-of-sale (POS) device” may refer to one or more devices, which may be used by a merchant to conduct a transaction (e.g., a payment transaction) and / or process a transaction. For example, a POS device may include one or more client devices. Additionally or alternatively, a POS device may include peripheral devices, card readers, scanning devices (e.g., code scanners), Bluetooth® communication receivers, near-field communication (NFC) receivers, RFID receivers, and / or other contactless transceivers or receivers, contact-based receivers, payment terminals, and / or the like. As used herein, a “point-of-sale (POS) system” may refer to one or more client devices and / or peripheral devices used by a merchant to conduct a transaction. For example, a POS system may include one or more POS devices and / or other like devices that may be used to conduct a payment transaction. In some non-limiting embodiments or aspects, a POS system (e.g., a merchant POS system) may include one or more server computers configured to process online payment transactions through webpages, mobile applications, and / or the like.

[0051] As used herein, the term “transaction service provider” may refer to an entity that receives transaction authorization requests from merchants or other entities and provides guarantees of payment, in some cases through an agreement between the transaction service provider and an issuer institution. For example, a transaction service provider may include a payment network such as Visa® or any other entity that processes transactions. The term “transaction processing system” may refer to one or more computer systems operated by or on behalf of a transaction service provider, such as a transaction processing server executing one or more software applications. A transaction processing server may include one or more processors and, in some non-limiting embodiments or aspects, may be operated by or on behalf of a transaction service provider.

[0052] Non-limiting embodiments or aspects of the disclosed subject matter are directed to methods, systems, and computer program products for time series analysis using a time interval embedding based attention mechanism. The disclosed subject matter provides for receiving training data associated with a time series of a plurality of events, determining a plurality of time intervals between each pair of events of the plurality of events of the time series, generating a time interval embedding for eachtime interval of the plurality of time intervals to provide a plurality of time interval embeddings, calculating a plurality of attention weights for an attention mechanism of a machine learning model based on the plurality of time interval embeddings, and generating a trained machine learning model based on the attention weights for the attention mechanism of the machine learning model. In some non-limiting embodiments or aspects, the machine learning model (e.g., the trained machine learning model) may include a recurrent neural network (RNN) machine learning model.

[0053] In some non-limiting embodiments or aspects, generating the time interval embedding for each time interval of the plurality of time intervals to provide the plurality of time interval embeddings may include determining an amount of time involved in each time interval of the plurality of time intervals, determining a discrete time interval of a plurality of discrete time intervals, and generating the time interval embedding for each time interval of the plurality of time intervals based on the discrete time interval associated with each time interval of the plurality of time intervals. In some non-limiting embodiments or aspects, the disclosed subject matter further provides for discretizing each time interval of the plurality of time intervals based on the amount of time involved in each time interval to provide the plurality of discrete time intervals.

[0054] In some non-limiting embodiments or aspects, the disclosed subject matter further provides for determining whether to perform an action based on an inference of the trained machine learning model. In some non-limiting embodiments or aspects, the disclosed subject matter further provides for calculating a plurality of attention scores of the machine learning model based on the plurality of time interval embeddings and the plurality of events, where each attention score comprises an output of the attention mechanism of the machine learning model based on an input provided to the attention mechanism. In some non-limiting embodiments or aspects, calculating the plurality of attention weights for the attention mechanism of the machine learning model may include calculating the plurality of attention weights for the attention mechanism of the machine learning model based on the plurality of attention scores of the machine learning model.

[0055] In some non-limiting embodiments or aspects, calculating the plurality of attention weights for the attention mechanism of the machine learning model may include executing a Tanh alignment function based on the plurality of time interval embeddings and the plurality of events to provide a time-embedded alignment factor,determining a first plurality of attention weights for the attention mechanism based on the plurality of time interval embeddings and the plurality of events, and multiplying the first plurality of attention weights for the attention mechanism by the time-embedded alignment factor to provide the plurality of attention weights for the attention mechanism of the machine learning model.

[0056] In this way, the disclosed subject matter may provide a machine learning model, such as an RNN machine learning model, that is configured to account for an amount of time in a time interval between events in a time series, such as uneven amounts of time in time intervals between events in the time series. Accordingly, the disclosed subject matter may provide for accurately performing analysis on time series data that has unevenly spaced time intervals between events in the time series. Further, the disclosed subject matter may provide increased accuracy regarding predictions, as compared to existing systems that do not involve the use of a plurality of attention weights for an attention mechanism of a machine learning model that is based on a plurality of time interval embeddings. Additionally, the disclosed subject matter is unique and unconventional.

[0057] For the purpose of illustration, in the following description, while the presently disclosed subject matter is described with respect to methods, systems, and computer program products for time series analysis using a time interval embedding based attention mechanism, e.g., for analysis of characteristics of electronic payment transactions represented as a time series, one skilled in the art will recognize that the disclosed subject matter is not limited to the non-limiting embodiments or aspects disclosed herein. For example, the methods, systems, and computer program products described herein may be used with a wide variety of settings, such as a machine learning model that is used to analyze sequence data (e.g., time series data) and / or for making determinations (e.g., predictions, classifications, regressions, and / or the like) with at least one machine learning model based on the sequential dataset, such as for fraud detection / prevention, authorization, authentication, identification, feature selection, product recommendation, stand in processing (STIP) of electronic payment transactions (e.g., a situation in which a transaction service provider acts in place of an issuer system to approve or deny an electronic payment transaction involving a customer account), and / or the like.

[0058] Referring now to FIG. 1 , shown is example system 100 for time series analysis using a time interval embedding based attention mechanism, according tosome non-limiting embodiments or aspects. For example, system 100 may include series analysis system 102, machine learning (ML) model management database 104, user device 106, and / or communication network 108.

[0059] Series analysis system 102 may include one or more devices capable of receiving information from and / or communicating information to ML model management database 104 and / or user device 106 (e.g., directly via wired or wireless communication connection, indirectly via communication network 108, and / or the like). For example, series analysis system 102 may include a computing device, such as a server, a group of servers, a desktop computer, a portable computer, a mobile device, and / or other like devices. In some non-limiting embodiments or aspects, series analysis system 102 may be in communication with a data storage device (e.g., ML model management database 104), which may be local or remote to series analysis system 102. In some non-limiting embodiments or aspects, series analysis system 102 may be capable of receiving information from, storing information in, communicating information to, or searching information stored in the data storage device (e.g., ML model management database 104).

[0060] ML model management database 104 may include one or more devices capable of receiving information from and / or communicating information to series analysis system 102 and / or user device 106 (e.g., directly via wired or wireless communication connection, indirectly via communication network 108, and / or the like). For example, ML model management database 104 may include a computing device, such as a server, a group of servers, a desktop computer, a portable computer, a mobile device, and / or other like devices. In some non-limiting embodiments or aspects, ML model management database 104 may include a data storage device. In some non-limiting embodiments or aspects, ML model management database 104 may be capable of receiving information from, storing information in, communicating information to, or searching information stored in the data storage device. In some non-limiting embodiments or aspects, ML model management database 104 may be part of series analysis system 102 and / or part of the same system as series analysis system 102.

[0061] User device 106 may include one or more devices capable of receiving information from and / or communicating information to series analysis system 102 and / or ML model management database 104 (e.g., directly via wired or wireless communication connection, indirectly via communication network 108, and / or the like).For example, user device 106 may include a computing device, such as a mobile device, a portable computer, a desktop computer, and / or other like devices. Additionally or alternatively, each user device 106 may include a device capable of receiving information from and / or communicating information to other user devices 106 (e.g., directly via wired or wireless communication connection, indirectly via communication network 108, and / or the like). In some non-limiting embodiments or aspects, user device 106 may be part of series analysis system 102 and / or part of the same system as series analysis system 102. For example, series analysis system 102, ML model management database 104, and user device 106 may all be (and / or be part of) a single system and / or a single computing device.

[0062] Communication network 108 may include one or more wired and / or wireless networks. For example, communication network 108 may include a cellular network (e.g., a long-term evolution (LTE®) network, a third generation (3G) network, a fourth generation (4G) network, a fifth generation (5G) network, a code division multiple access (CDMA) network, and / or the like), a public land mobile network (PLMN), a local area network (LAN), a wide area network (WAN), a metropolitan area network (MAN), a telephone network (e.g., the public switched telephone network (PSTN)), a private network (e.g., a private network associated with a transaction service provider), an ad hoc network, an intranet, the Internet, a fiber optic-based network, a cloud computing network, and / or the like, and / or a combination of these or other types of networks.

[0063] The number and arrangement of systems and devices shown in FIG. 1 are provided as an example. There may be additional systems and / or devices, fewer systems and / or devices, different systems and / or devices, and / or differently arranged systems and / or devices than those shown in FIG. 1. Furthermore, two or more systems or devices shown in FIG. 1 may be implemented within a single system or device, or a single system or device shown in FIG. 1 may be implemented as multiple, distributed systems or devices. Additionally or alternatively, a set of systems (e.g., one or more systems) or a set of devices (e.g., one or more devices) of system 100 may perform one or more functions described as being performed by another set of systems or another set of devices of system 100.

[0064] Referring now to FIG. 2, shown is a flow diagram for process 200 for generating a trained machine learning model that includes a time interval embedding based attention mechanism, according to some non-limiting embodiments or aspects. The steps shown in FIG. 2 are for example purposes only. It will be appreciated thatadditional, fewer, different, and / or different order of steps may be used in non-limiting embodiments or aspects. In some non-limiting embodiments or aspects, a step may be automatically performed in response to performance and / or completion of a prior step. In some non-limiting embodiments or aspects, process 200 may be performed during a training process. In some non-limiting embodiments or aspects, one or more of the steps of process 200 may be performed (e.g., completely, partially, and / or the like) by series analysis system 102 (e.g., at least one computing device of series analysis system 102). In some non-limiting embodiments or aspects, one or more of the steps of process 200 may be performed (e.g., completely, partially, and / or the like) by another system, another device, another group of systems, or another group of devices, separate from or including series analysis system 102, such as ML model management database 104, user device 106, and / or the like.

[0065] As shown in FIG. 2, at step 202, process 200 may include determining a plurality of time intervals between each pair of events of a plurality of events of a time series. For example, series analysis system 102 may determine the plurality of time intervals (e.g., gaps of time) between each pair of events of a plurality of events of the time series. In some non-limiting embodiments or aspects, series analysis system 102 may generate a timestamp (e.g., an indication of an amount of time of a time interval) between each pair of events of a plurality of events of the time series.

[0066] In some non-limiting embodiments or aspects, series analysis system 102 may receive data (e.g., training data) associated with a time series of a plurality of events. In some non-limiting embodiments or aspects, the data associated with the time series of the plurality of events may include data associated with a time series (e.g., a time ordered sequence) of electronic payment transactions involving an account of a customer. The data associated with the time series of electronic payment transactions may include transaction data associated with the plurality of electronic payment transactions.

[0067] In some non-limiting embodiments or aspects, the training dataset may include a plurality of data instances (e.g., a plurality of data instances that are labeled correctly with ground truth values) associated with a time series of electronic payment transactions. In some non-limiting embodiments or aspects, each data instance may have a label, such as a label indicating whether the data instance is associated with a non-fraudulent data instance (e.g., a data instance associated with a non-fraudulent payment transaction) or a fraudulent data instance (e.g., a data instance associatedwith a fraudulent payment transaction), and a label indicating whether the data instance is associated with a data instance that was authorized (e.g., an electronic payment transaction that was approved or an electronic payment transaction that was declined).

[0068] In some non-limiting embodiments or aspects, a data instance that is labeled correctly may include a data instance that is a verified (e.g., reported as, reported by entity of a payment network as, such as reported by ML model management database 104, user device 106, etc.) electronic payment transaction that was approved or an electronic payment transaction that was declined.

[0069] In some examples, the training dataset may include a large amount of data instances, such as 100 data instances, 500 data instances, 1 ,000 data instances, 5,000 data instances, 10,000 data instances, 25,000 data instances, 50,000 data instances, 100,000 data instances, 1 ,000,000 data instances, and / or the like. In some non-limiting embodiments or aspects, a percentage (e.g., a first percentage) of the plurality of data instances are labeled correctly (e.g., labeled correctly with a positive label of a binary classification, labeled correctly with a negative label of a binary classification, etc.). In some non-limiting embodiments or aspects, the plurality of data instances is labeled based on labels provided as an output of a classification machine learning model (e.g., a deep learning fraud detection model, such as a fraud detection model that includes a classification machine learning model). In some non-limiting embodiments or aspects, the training dataset may include a plurality of historical data instances. In some non-limiting embodiments or aspects, the training dataset may include data (e.g., transaction data) associated with historical electronic payment transactions that were conducted using one or more payment processing networks (e.g., one or more payment processing networks associated with ML model management database 104).

[0070] In some non-limiting embodiments or aspects, the training dataset may include a plurality of data instances associated with a plurality of features. In some examples, the plurality of data instances may include a large number of features (e.g., in a range), such as 50 features, 100 features, 500 features, 1 ,000 features, 5,000 features, 10,000 features, 100,000 features, and / or the like. In some non-limiting embodiments or aspects, the plurality of data instances may represent a plurality of transactions (e.g., electronic payment transactions) conducted by one or moreaccountholders (e.g., one or more users, such as a user associated with user device 106).

[0071] In some non-limiting embodiments or aspects, a data instance (e.g., a data instance of a dataset, such as an initial training dataset, a second training dataset, or a testing dataset) may include transaction data associated with a payment transaction. In some non-limiting embodiments or aspects, the transaction data may include a plurality of transaction parameters associated with an electronic payment transaction. In some non-limiting embodiments or aspects, the plurality of features may represent the plurality of transaction parameters. In some non-limiting embodiments or aspects, the plurality of transaction parameters may include electronic wallet card data associated with an electronic card (e.g., an electronic credit card, an electronic debit card, an electronic loyalty card, and / or the like), decision data associated with a decision (e.g., a decision to approve or deny a transaction authorization request), authorization data associated with an authorization response (e.g., an approved spending limit, an approved transaction value, and / or the like), a PAN, an authorization code (e.g., a personal identification number (PIN), etc.), data associated with a transaction amount (e.g., an approved limit, a transaction value, etc.), data associated with a transaction date and / or time (e.g., data associated with a date and / or time at which an electronic payment transaction was initiated, data associated with a date and / or time at which an electronic payment transaction was authorized or not authorized, data associated with a date and / or time at which an electronic payment transaction was settled, data associated with a date and / or time at which an electronic payment transaction was cleared, etc.), data associated with a conversion rate of a currency, data associated with a merchant type (e.g., a merchant category code that indicates a type of goods, such as grocery, fuel, and / or the like), data associated with an acquiring institution country, data associated with an identifier of a country associated with the PAN, data associated with a response code, data associated with a merchant identifier (e.g., a merchant name, a merchant location, and / or the like), data associated with a type of currency corresponding to funds stored in association with the PAN, and / or the like.

[0072] In some non-limiting embodiments or aspects, the data associated with a time series of a plurality of events may include time series data associated with a date and / or time of a plurality of sequential events, such as a sequence of electronic payment transactions during a time period and for a specified account.

[0073] In some non-limiting embodiments or aspects, series analysis system 102 may determine a plurality of time intervals between each pair of events of the plurality of events of the time series based on the data associated with the time series of the plurality of events. In some non-limiting embodiments or aspects, series analysis system 102 may determine the plurality of time intervals based on receiving the data associated with the time series of the plurality of events. In some non-limiting embodiments or aspects, series analysis system 102 may determine the plurality of time intervals by comparing timestamps for each pair of events of the plurality of events of the time series and determining a difference between each timestamp.

[0074] As shown in FIG. 2, at step 204, process 200 may include generating a time interval embedding for each time interval of the plurality of time intervals. For example, series analysis system 102 may generate the time interval embedding for each time interval of the plurality of time intervals based on determining the plurality of time intervals between each pair of events of the plurality of events of the time series.

[0075] In some non-limiting embodiments or aspects, series analysis system 102 may determine an amount of time involved in each time interval of the plurality of time intervals, determine a discrete time interval of a plurality of discrete time intervals, and generate the time interval embedding for each time interval of the plurality of time intervals based on the discrete time interval associated with each time interval of the plurality of time intervals. In some non-limiting embodiments or aspects, series analysis system 102 may discretize each time interval of the plurality of time intervals based on the amount of time involved in each time interval to provide the plurality of discrete time intervals. In this way, series analysis system 102 may reduce resources necessary for generating the time interval embeddings by providing a range of an amount of time for each time interval of the plurality of time intervals that is based on the plurality of time intervals between each pair of events of the plurality of events of the time series as compared to providing a discrete point for an amount of time for each time interval of the plurality of time intervals. In some non-limiting embodiments or aspects, one or more time intervals of the plurality of time intervals may be uneven (e.g., unequal) as compared to another time interval of the plurality of time intervals.

[0076] As shown in FIG. 2, at step 206, process 200 may include calculating a plurality of attention weights for an attention mechanism of a machine learning model. For example, series analysis system 102 may calculate a plurality of attention weights for an attention mechanism of the machine learning model based on the plurality oftime interval embeddings. In some non-limiting embodiments or aspects, the machine learning model may include an RNN, such as a long short-term memory (LSTM) machine learning model and a gated recurrent unit (GRU) machine learning model.

[0077] In some non-limiting embodiments or aspects, series analysis system 102 may calculate a plurality of attention scores of the machine learning model based on the plurality of time interval embeddings and the plurality of events. In some nonlimiting embodiments or aspects, each attention score may include an output of the attention mechanism of the machine learning model based on an input provided to the attention mechanism. In some non-limiting embodiments or aspects, series analysis system 102 may calculate the plurality of attention weights for the attention mechanism of the machine learning model based on the plurality of attention scores of the machine learning model.

[0078] In some non-limiting embodiments or aspects, series analysis system 102 may provide data associated with a time series of a plurality of events as an input to an RNN, and may determine a hidden state (e.g., a representation of a hidden state) for each event of the plurality of events to provide a plurality of hidden states. Series analysis system 102 may then calculate a plurality of attention scores for the plurality of hidden states, for example, by applying a scoring function (e.g., a learnable scoring function with trainable parameters) to each hidden state of the plurality of hidden states. In some non-limiting embodiments or aspects, series analysis system 102 may normalize the plurality of attention scores into a plurality of attention weights for the attention mechanism of the machine learning model. For example, series analysis system 102 may apply a softmax function to the plurality of attention scores (e.g., provide each attention score as an input to the softmax function) to provide the plurality of attention weights for the attention mechanism. In some non-limiting embodiments or aspects, series analysis system 102 may compute a context vector for the attention mechanism. For example, series analysis system 102 may compute a context vector based on a weighted sum of the plurality of hidden states.

[0079] In some non-limiting embodiments or aspects, when calculating the plurality of attention weights for the attention mechanism of the machine learning model, series analysis system 102 may execute a Tanh alignment function based on the plurality of time interval embeddings and the plurality of events to provide a time-embedded alignment factor, determine a first plurality of attention weights for the attention mechanism based on the plurality of time interval embeddings and the plurality ofevents, and multiply the first plurality of attention weights for the attention mechanism by the time-embedded alignment factor to provide the plurality of attention weights for the attention mechanism of the machine learning model.

[0080] As shown in FIG. 2, at step 208, process 200 may include generating a trained machine learning model. For example, series analysis system 102 may generate the trained machine learning model based on the attention weights for the attention mechanism of the machine learning model. In some non-limiting embodiments or aspects, series analysis system 102 may generate the trained machine learning model based on backpropagation of the attention weights for the attention mechanism of the machine learning model.

[0081] During training, series analysis system 102 may train the machine learning model by providing a sequence of events (e.g., data associated with one or more time series of a plurality of events to form the training dataset) as an input to the machine learning model and generating a hidden state for each event. In some non-limiting embodiments or aspects, series analysis system 102 may then calculate attention scores for each hidden state and compute attention weights for an attention mechanism based on the attention scores. In some non-limiting embodiments or aspects, series analysis system 102 may then compute a context vector for the attention mechanism. In some non-limiting embodiments or aspects, series analysis system 102 may then generate outputs (e.g., output predictions) based on the training dataset provided as inputs, determine a loss calculation using a loss function, and backpropagate an error amount based on the loss calculation. Series analysis system 102 may then update parameters of the machine learning model and / or parameters of the attention mechanism until the training process is complete.

[0082] In some non-limiting embodiments or aspects, series analysis system 102 may determine whether to perform an action (e.g., an action associated with authorization of an electronic payment transaction, an action associated with determining whether an electronic payment transaction is fraudulent, etc.) based on an inference of the trained machine learning model. For example, series analysis system 102 may determine to perform an action based on the inference of the trained machine learning model being equal to a first value. In such an example, series analysis system 102 may determine to forgo performing an action based on the inference of the trained machine learning model being equal to a second value (e.g., a second value that is different from the first value).

[0083] In some non-limiting embodiments or aspects, series analysis system 102 may receive a request for inference for a machine learning model (e.g., trained machine learning model that has been placed into a production or runtime environment) and series analysis system 102 may generate an inference based on the request. In some non-limiting embodiments or aspects, the machine learning model may include a machine learning model that has been trained and / or validated (e.g., tested) and that may be used to generate inferences (e.g., predictions), such as real-time inferences, runtime inferences, and / or the like. In some non-limiting embodiments or aspects, a production machine learning model may include the updated trained machine learning model.

[0084] In some non-limiting embodiments or aspects, the request for inference may be associated with a task for which the machine learning model may provide an inference. In some non-limiting embodiments or aspects, the request for inference may be associated with financial service tasks. For example, the request for inference may be associated with a token service task, an authentication task (e.g., a 3D secure authentication task), a fraud detection task, and / or the like.

[0085] In some non-limiting embodiments or aspects, the request for inference may include runtime input data. In some non-limiting embodiments or aspects, the runtime input data may include a sample of data that is received by a trained machine learning model in real-time with respect to the runtime input data being generated. For example, runtime input data may be generated by a data source (e.g., a customer performing a transaction) and may be subsequently received by the trained machine learning model in real-time. Runtime (e.g., production) may refer to inputting runtime data (e.g., a runtime dataset, real-world data, real-world observations, and / or the like) into one or more trained machine learning models (e.g., one or more trained machine learning models of series analysis system 102) and / or generating an inference (e.g., generating an inference using series analysis system 102 or another system).

[0086] In some non-limiting embodiments or aspects, runtime may be performed during a phase which may occur after a training phase, after a testing phase, and / or after deployment of the machine learning model into a production environment. During a time period associated with the runtime phase, the machine learning model (e.g., a production machine learning model) may process the runtime input data to generate inferences (e.g., real-time inferences, real-time predictions, and / or the like).

[0087] Referring now to FIGS. 3A-3D, shown are schematic diagrams of implementation 300 of a process (e.g., process 200) for generating a trained machine learning model that includes a time interval embedding based attention mechanism. In some non-limiting embodiments or aspects, one or more of the steps of the process may be performed (e.g., completely, partially, etc.) by series analysis system 102 (e.g., one or more devices of series analysis system 102). In some non-limiting embodiments or aspects, one or more of the steps of the process may be performed (e.g., completely, partially, etc.) by another device or a group of devices separate from or including series analysis system 102 (e.g., one or more devices of series analysis system 102), ML model management database 104, and / or user device 106.

[0088] As shown by reference number 305 in FIG. 3A, series analysis system 102 may receive a training dataset from ML model management database 104. For example, series analysis system 102 may receive a training dataset that includes data associated with a time series of a plurality of electronic payment transactions (e.g., a time ordered sequence of electronic payment transactions involving an account of a customer) and an authorization status of each of the plurality of electronic payment transactions (e.g., shown as “Declined”, where the status is shown as a binary result, with “0” representing an electronic payment transaction that was authorized and “1 ” representing an electronic payment transaction that was not authorized). As further shown by reference number 310 in FIG. 3A, series analysis system 102 may determine a plurality of time intervals between each pair of events of a plurality of events of a time series, such as each pair of electronic payment transactions. For example, series analysis system 102 may determine a plurality of time intervals (e.g., gaps of time) between each pair of electronic payment transactions by comparing timestamps (e.g., shown as an authorization time of an electronic payment transaction) for each pair of events of the plurality of events of the time series and determining a difference between each timestamp.

[0089] As shown by reference number 315 in FIG. 3B, series analysis system 102 may generate a time interval embedding for each time interval of the plurality of time intervals. In some non-limiting embodiments or aspects, series analysis system 102 may configure a plurality of discrete time intervals (e.g., shown as “Seg” for segments), each of which correspond to an associated time interval embedding. For example, series analysis system 102 may configure a first discrete time interval (e.g., shown as Seg “0”) based on a time interval that is less than 1 minute (e.g., 1 second toapproximately 60 seconds), a second discrete time interval (e.g., shown as Seg “1 ”) based on a time interval that is in a range of 1 minute to approximately 1 hour (e.g., 60 seconds to approximately 3,600 seconds), a third discrete time interval (e.g., shown as Seg “2”) based on a time interval that is in a range of 1 hour to approximately 10 hours (e.g., 3,600 seconds to approximately 36,000 seconds), a fourth discrete time interval (e.g., shown as Seg “3”) based on a time interval that is in a range of 10 hours to approximately 1 day (e.g., 36,000 seconds to approximately 86,400 seconds), a fifth discrete time interval (e.g., shown as Seg “4”) based on a time interval that is in a range of 1 day to approximately 5 days (e.g., 84,600 seconds to approximately 432,000 seconds), and a sixth discrete time interval (e.g., shown as Seg “5”) based on a time interval that is greater than 5 days (e.g., greater than 432,000 seconds).

[0090] In some non-limiting embodiments or aspects, series analysis system 102 may determine an amount of time involved in each time interval of the plurality of time intervals, determine a discrete time interval of a plurality of discrete time intervals, and generate the time interval embedding for each time interval of the plurality of time intervals based on the discrete time interval associated with each time interval of the plurality of time intervals. In some non-limiting embodiments or aspects, series analysis system 102 may generate the plurality of time interval embeddings based on a machine learning model (e.g., an RNN machine learning model). For example, series analysis system 102 may provide the training dataset (e.g., a portion of the training dataset) as an input to the machine learning model to generate the plurality of time interval embeddings as embeddings of the machine learning model. In some nonlimiting embodiments or aspects, series analysis system 102 may generate the plurality of time interval embeddings before training a machine learning model, and series analysis system 102 may obtain a time interval embedding for each discrete time interval based on a lookup table.

[0091] As shown by reference number 320 in FIG. 3C, series analysis system 102 may calculate a plurality of attention weights for an attention mechanism of an RNN machine learning model. For example, series analysis system 102 may provide an attention input (e.g., a vector that is provided by the last layer of the RNN machine learning model, such as a stacked LSTM or GRU), which comprises a hidden state for each electronic payment transaction of the plurality of electronic payment transactions as an input to an attention mechanism, where strepresents a hidden state of the RNN machine learning model for a most recent electronic payment transaction, and Krepresents a total number of electronic payment transactions. In the example, series analysis system 102 may calculate a plurality of attention scores for each electronic payment transaction based on a hidden state of the RNN machine learning model and a specified hidden state, sk.

[0092] Further, series analysis system 102 may determine the plurality of attention weights for the attention mechanism based on the formula:

[0094] for k = t — 1, t — (K — 1), where alk= align ek, sk~) = wj * Tanh (W±* [ek,sk]) and alkis a Tanh alignment function, and [ek,sk] is the concatenation of the vectors ekand sk. In addition, w2and 14^ are trainable weight matrices of the Tanh alignment function. If the length of ekand skare Leand Ls, then the shapes of w2and 14^ are L * 1 and L * (Le+ LS), L may be a predetermined dimension for the Tanh alignment function (e.g., 50 or 100). In some non-limiting embodiments or aspects, the embedding size, Le, may be in a range of 1 -20, or in specific examples, 5 or 10.

[0095] An output of the attention mechanism may be calculated based on the formula:

[0097] The attention weights and the trainable weight matrices of the Tanh alignment function may be updated during training of the RNN machine learning model.

[0098] As shown by reference number 325 in FIG. 3D, series analysis system 102 may train the RNN machine learning model to provide a trained RNN machine learning model.

[0099] Referring now to FIG. 4, depicted is a diagram of example payment processing network 400, according to non-limiting embodiments or aspects. In some non-limiting embodiments or aspects, payment processing network 400 may be used in conjunction with the methods, systems, and / or computer program products described herein, and / or the methods, systems, and / or computer program products described herein may be implemented in payment processing network 400. As shown in FIG. 4, payment processing network 400 may include transaction processing system 402, payment gateway system 412, merchant system 408, issuer system 404, acquirer system 410, and / or customer device 406. In some non-limiting embodiments or aspects, each of series analysis system 102, ML model management database 104,and / or user device 106 of FIG. 1 may be implemented by (e.g., part of) transaction processing system 402. In some non-limiting embodiments or aspects, at least one of series analysis system 102, ML model management database 104, and / or user device 106 of FIG. 1 may be implemented by (e.g., part of) another system, another device, another group of systems, or another group of devices, separate from or including transaction processing system 402, such as merchant system 408, issuer system 404, acquirer system 410, customer device 406, and / or the like. For example, series analysis system 102 may be implemented by (e.g., part of) at least one of payment gateway system 412, merchant system 408, issuer system 404, acquirer system 410, and / or customer device 406. The systems and / or devices of FIG. 4 may communicate via communication network 414, which may include one or more wired and / or wireless communication networks.

[0100] Transaction processing system 402 may include one or more devices capable of receiving information from and / or communicating information to payment gateway system 412, merchant system 408, issuer system 404, acquirer system 410, customer device 406, and / or the like (e.g., directly, indirectly, via a public and / or private communication network connection, and / or the like). For example, as shown in FIG. 4, transaction processing system 402 may be in communication with one or more issuer systems (e.g., issuer system 404), one or more acquirer systems (e.g., acquirer system 410), and / or one or more payment gateway systems (e.g., payment gateway system 412). Although only a single issuer system 404, a single acquirer system 410, and a single payment gateway system 412 are shown, it will be appreciated that transaction processing system 402 may be in communication with a plurality of issuer systems, a plurality of acquirer systems, and / or a plurality of payment gateway systems. In some non-limiting embodiments or aspects, transaction processing system 402 may include a computing device, such as a server (e.g., a transaction processing server), a group of servers, and / or other like devices. In some non-limiting embodiments or aspects, transaction processing system 402 may be in communication with a data storage device, which may be local or remote to transaction processing system 402. In some non-limiting embodiments or aspects, transaction processing system 402 may be capable of receiving information from, storing information in, communicating information to, or searching information stored in the data storage device. In some non-limiting embodiments or aspects, transaction processing system 402 may be associated with a transaction service provider, asdescribed herein. In some non-limiting embodiments or aspects, transaction processing system 402 may also operate as an issuer system, such that both transaction processing system 402 and issuer system 404 are a single system and / or are controlled by a single entity.

[0101] Payment gateway system 412 may include one or more devices capable of receiving information from and / or communicating information to transaction processing system 402, merchant system 408, issuer system 404, acquirer system 410, customer device 406, and / or the like (e.g., directly, indirectly, via a public and / or private communication network connection, and / or the like). For example, as shown in FIG. 4, payment gateway system 412 may be in communication with one or more merchant systems (e.g., merchant system 408), one or more acquirer systems (e.g., acquirer system 410), and / or one or more transaction processing systems (e.g., transaction processing system 402). Although only a single merchant system 408, a single acquirer system 410, and a single transaction processing system 402 are shown, it will be appreciated that payment gateway system 412 may be in communication with a plurality of merchant systems, a plurality of acquirer systems, and / or a plurality of transaction processing systems. In some non-limiting embodiments or aspects, payment gateway system 412 may include a computing device, such as a server, a group of servers, and / or other like devices. In some nonlimiting embodiments or aspects, payment gateway system 412 may be associated with a payment gateway, as described herein.

[0102] Merchant system 408 may include one or more devices capable of receiving information from and / or communicating information to transaction processing system 402, payment gateway system 412, issuer system 404, acquirer system 410, customer device 406, and / or the like (e.g., directly, indirectly, via a public and / or private communication network connection, and / or the like). For example, as shown in FIG. 4, merchant system 408 may be in communication with one or more payment gateway systems (e.g., payment gateway system 412), one or more acquirer systems (e.g., acquirer system 410), and / or one or more consumer devices (e.g., customer device 406). Although only a single payment gateway system 412, a single acquirer system 410, and a single customer device 406 are shown, it will be appreciated that merchant system 408 may be in communication with a plurality of payment gateway systems, a plurality of acquirer systems, and / or a plurality of consumer devices. In some non-limiting embodiments or aspects, merchant system408 may include a computing device, such as a server, a group of servers, a client device, a group of client devices, a POS device, a POS system, computers, computer systems, peripheral devices, and / or other like devices. In some non-limiting embodiments or aspects, merchant system 408 may be associated with a merchant, as described herein. In some non-limiting embodiments or aspects, merchant system 408 may include a device capable of receiving information from and / or communicating information to customer device 406 via a short range communication connection (e.g., an NFC communication connection, an RFID communication connection, a Bluetooth® communication connection, a Zigbee® communication connection, and / or the like) with customer device 406 and / or the like. In some non-limiting embodiments or aspects, merchant system 408 may include one or more client devices. For example, merchant system 408 may include a client device that allows a merchant to communicate information to transaction processing system 402 (e.g., via at least one of acquirer system 410 and / or payment gateway system 412). In some non-limiting embodiments or aspects, merchant system 408 (e.g., a client device thereof, a POS device thereof, and / or the like) may also operate as a payment gateway system, such that both merchant system 408 and payment gateway system 412 are a single system and / or controlled by a single entity.

[0103] Issuer system 404 may include one or more devices capable of receiving information and / or communicating information to transaction processing system 402, payment gateway system 412, merchant system 408, acquirer system 410, customer device 406, and / or the like (e.g., directly, indirectly, via a public and / or private communication network connection, and / or the like). For example, as shown in FIG. 4, issuer system 404 may be in communication with one or more transaction processing systems (e.g., transaction processing system 402) and / or one or more consumer devices (e.g., customer device 406). Although only a single transaction processing system 402 and a single customer device 406 are shown, it will be appreciated that issuer system 404 may be in communication with a plurality of transaction processing systems and / or a plurality of customer devices 406. In some non-limiting embodiments or aspects, issuer system 404 may include a computing device, such as a server, a group of servers, and / or other like devices. In some non-limiting embodiments or aspects, issuer system 404 may be associated with an issuer institution, as described herein. For example, issuer system 404 may be associated with an issuer institutionthat issued a credit account, debit account, credit card, debit card, a payment device, and / or the like to a user associated with customer device 406.

[0104] Acquirer system 410 may include one or more devices capable of receiving information from and / or communicating information to transaction processing system 402, payment gateway system 412, merchant system 408, issuer system 404, customer device 406, and / or the like (e.g., directly, indirectly, via a public and / or private communication network connection, and / or the like). For example, as shown in FIG. 4, acquirer system 410 may be in communication with one or more transaction processing systems (e.g., transaction processing system 402), one or more payment gateway systems (e.g., payment gateway system 412), and / or one or more merchant systems (e.g., merchant system 408). Although only a single transaction processing system 402, a single payment gateway system 412, and a single merchant system 408 are shown, it will be appreciated that acquirer system 410 may be in communication with a plurality of transaction processing systems, a plurality of payment gateway systems, and / or a plurality of merchant systems. In some nonlimiting embodiments or aspects, acquirer system 410 may include a computing device, such as a server, a group of servers, and / or other like devices. In some nonlimiting embodiments or aspects, acquirer system 410 may be associated with an acquirer institution, as described herein.

[0105] Customer device 406 may include one or more devices capable of receiving information from and / or communicating information to transaction processing system 402, payment gateway system 412, merchant system 408, issuer system 404, acquirer system 410, and / or the like (e.g., directly, indirectly, via a public and / or private communication network connection, and / or the like). For example, as shown in FIG. 4, customer device 406 may be in communication with one or more merchant systems (e.g., merchant system 408) and / or one or more issuer systems (e.g., issuer system 404). Although only a single merchant system 408 and a single issuer system 404 are shown, it will be appreciated that customer device 406 may be in communication with a plurality of merchant systems and / or a plurality of issuer systems. In some non-limiting embodiments or aspects, customer device 406 may be associated with a user to whom a credit account, debit account, credit card, debit card, a payment device, and / or the like has been issued. In some non-limiting embodiments or aspects, customer device 406 may include a computing device, such as a computer, a portable computer, a laptop computer, a tablet computer, a mobile device, a cellularphone, a smartphone, a wearable device (e.g., watches, glasses, lenses, clothing, and / or the like), a PDA, a client device, and / or other like devices. In some non-limiting embodiments or aspects, customer device 406 may include a payment device, as described herein. In some non-limiting embodiments or aspects, customer device 406 may include a device capable of receiving information from and / or communicating information to other customer devices 406 (e.g., directly, indirectly, via a public and / or private communication network connection, a short range communication connection, and / or the like). In some non-limiting embodiments or aspects, customer device 406 may include a device capable of receiving information from and / or communicating information to merchant system 408 via a short range communication connection (e.g., an NFC communication connection, an RFID communication connection, a Bluetooth® communication connection, a Zigbee® communication connection, and / or the like) with merchant system 408 and / or the like. In some non-limiting embodiments or aspects, customer device 406 may include a client device.

[0106] In some non-limiting embodiments or aspects, transaction processing system 402 may communicate with merchant system 408 directly (e.g., via a public and / or private communication network connection and / or the like). Additionally or alternatively, transaction processing system 402 may communicate with merchant system 408 through payment gateway system 412 and / or acquirer system 410. In some non-limiting embodiments or aspects, acquirer system 410 associated with merchant system 408 may operate as payment gateway system 412 to facilitate the communication of transaction messages (e.g., authorization requests) from merchant system 408 to transaction processing system 402. In some non-limiting embodiments or aspects, merchant system 408 may communicate with payment gateway system 412 directly (e.g., via a public and / or private communication network connection and / or the like). For example, merchant system 408, that includes a physical POS device, may communicate with payment gateway system 412 through a public or private network to conduct card-present transactions. As another example, merchant system 408 that includes a server (e.g., a web server) may communicate with payment gateway system 412 through a public or private network, such as the Internet, to conduct card-not-present transactions.

[0107] For the purpose of illustration, processing a transaction (e.g., a payment transaction) may include generating a transaction message (e.g., authorization request and / or the like) based on an account identifier of a customer (e.g.,accountholder associated with customer device 406 and / or the like) and / or transaction data associated with the transaction. For example, merchant system 408 (e.g., a client device of merchant system 408, a POS device of merchant system 408, and / or the like) may initiate the transaction, e.g., by generating an authorization request (e.g., in response to receiving the account identifier from a payment device and / or a portable financial device of the customer and / or the like). Merchant system 408 may communicate the authorization request to payment gateway system 412 and / or acquirer system 410. In some non-limiting embodiments or aspects, payment gateway system 412 may communicate the authorization request to acquirer system 410 and / or transaction processing system 402. Additionally or alternatively, acquirer system 410 (and / or payment gateway system 412) may communicate the authorization request to transaction processing system 402. After receiving the authorization request from merchant system 408 that identifies the account identifier of the customer (e.g., the accountholder associated with customer device 406 and / or the account identifier), transaction processing system 402 may communicate the authorization request to issuer system 404 (e.g., the issuer system that issued the payment device and / or account identifier). Issuer system 404 may determine an authorization decision (e.g., approve, deny, and / or the like) based on the authorization request, and / or issuer system 404 may generate an authorization response based on the authorization decision and / or the authorization request. Issuer system 404 may communicate the authorization response to transaction processing system 402. Transaction processing system 402 may communicate the authorization response to acquirer system 410 and / or payment gateway system 412. In some non-limiting embodiments or aspects, acquirer system 410 may communicate the authorization response to payment gateway system 412 and / or merchant system 408. Additionally or alternatively, payment gateway system 412 (and / or acquirer system 410) may communicate the authorization response to merchant system 408.

[0108] In some non-limiting embodiments or aspects, transaction processing system 402 and / or issuer system 404 may include at least one machine learning model (e.g., at least one of a fraud detection model, a risk detection model, a transaction authorization model, a credit approval model, a product recommendation model, a classifier model, an anomaly detection model, an authentication model, any combination thereof, and / or the like). For example, the machine learning model(s) may be trained based on synthetic data generated, as described herein. Transactionprocessing system 402 and / or issuer system 404 may perform at least one task (e.g., generate a prediction and / or generate an embedding) based on the authorization request and the machine learning model(s). For example, performing the task(s) may include generating at least one prediction associated with fraud detection, risk detection, transaction authorization, credit approval, product recommendation, classification, anomaly detection, authentication, any combination thereof, and / or the like. In some non-limiting embodiments or aspects, transaction processing system 402 may communicate at least one message based on performing the task (e.g., generating the prediction and / or generating an embedding) to issuer system 404 (e.g., along with the authorization request). In some non-limiting embodiments or aspects, issuer system 404 may determine the authorization decision (e.g., approve, deny, and / or the like) based on the authorization request and the performance of the task (e.g., generation of the prediction and / or generation of the embedding).

[0109] For the purpose of illustration, clearing and / or settlement of a transaction may include generating a message (e.g., clearing message and / or the like) based on an account identifier of a customer (e.g., associated with customer device 406 and / or the like) and / or transaction data associated with the transaction. For example, merchant system 408 may generate at least one clearing message (e.g., a plurality of clearing messages, a batch of clearing messages, and / or the like). Merchant system 408 may communicate the clearing message(s) to acquirer system 410 (and / or payment gateway system 412, which may communicate the clearing message(s) to acquirer system 410). Acquirer system 410 may communicate the clearing message(s) to transaction processing system 402. Transaction processing system 402 may communicate the clearing message(s) to issuer system 404. Issuer system 404 may generate at least one settlement message based on the clearing message(s). In some non-limiting embodiments or aspects, issuer system 404 may communicate the settlement message(s) and / or funds to transaction processing system 402 (and / or a settlement bank system associated with transaction processing system 402), and transaction processing system 402 (and / or the settlement bank system) may communicate the settlement message(s) and / or funds to acquirer system 410. Additionally or alternatively, issuer system 404 may communicate the settlement message(s) and / or funds to acquirer system 410. In some non-limiting embodiments or aspects, acquirer system 410 may communicate the settlement message(s) and / orfunds to merchant system 408 (and / or an account associated with merchant system 408).

[0110] Communication network 414 may include one or more wired and / or wireless networks. For example, communication network 414 may include a cellular network (e.g., a long-term evolution (LTE) network, a third generation (3G) network, a fourth generation (4G) network, a fifth generation (5G) network, a code division multiple access (CDMA) network, and / or the like), a public land mobile network (PLMN), a local area network (LAN), a wide area network (WAN), a metropolitan area network (MAN), a telephone network (e.g., the public switched telephone network (PSTN)), a private network (e.g., a private network associated with a transaction service provider), an ad hoc network, an intranet, the Internet, a fiber optic-based network, a cloud computing network, and / or the like, and / or a combination of these or other types of networks.

[0111] The number and arrangement of systems, devices, and / or networks shown in FIG. 4 are provided as an example. There may be additional systems, devices, and / or networks; fewer systems, devices, and / or networks; different systems, devices, and / or networks; and / or differently arranged systems, devices, and / or networks than those shown in FIG. 4. Furthermore, two or more systems or devices shown in FIG. 4 may be implemented within a single system or device, or a single system or device shown in FIG. 4 may be implemented as multiple, distributed systems or devices. Additionally or alternatively, a set of systems (e.g., one or more systems) or a set of devices (e.g., one or more devices) of environment 400 may perform one or more functions described as being performed by another set of systems or another set of devices of environment 400.

[0112] Referring now to FIG. 5, shown is a diagram of example components of device 500, according to non-limiting embodiments or aspects. Device 500 may correspond to at least one of series analysis system 102, ML model management database 104, and / or user device 106 in FIG. 1 and / or at least one of transaction processing system 402, issuer system 404, customer device 406, merchant system 408, and / or acquirer system 410 in FIG. 4, as an example. In some non-limiting embodiments or aspects, such systems or devices in FIG. 1 or FIG. 4 may include at least one device 500 and / or at least one component of device 500. The number and arrangement of components shown in FIG. 5 are provided as an example. In some non-limiting embodiments or aspects, device 500 may include additional components,fewer components, different components, or differently arranged components than those shown in FIG. 5. Additionally or alternatively, a set of components (e.g., one or more components) of device 500 may perform one or more functions described as being performed by another set of components of device 500.

[0113] As shown in FIG. 5, device 500 may include bus 502, processor 504, memory 506, storage component 508, input component 510, output component 512, and communication interface 514. Bus 502 may include a component that permits communication among the components of device 500. In some non-limiting embodiments or aspects, processor 504 may be implemented in hardware, firmware, or a combination of hardware and software. For example, processor 504 may include a processor (e.g., a central processing unit (CPU), a graphics processing unit (GPU), an accelerated processing unit (APU), etc.), a microprocessor, a digital signal processor (DSP), and / or any processing component (e.g., a field-programmable gate array (FPGA), an application-specific integrated circuit (ASIC), etc.) that can be programmed to perform a function. Memory 506 may include random access memory (RAM), read only memory (ROM), and / or another type of dynamic or static storage device (e.g., flash memory, magnetic memory, optical memory, etc.) that stores information and / or instructions for use by processor 504.

[0114] With continued reference to FIG. 5, storage component 508 may store information and / or software related to the operation and use of device 500. For example, storage component 508 may include a hard disk (e.g., a magnetic disk, an optical disk, a magneto-optic disk, a solid-state disk, etc.) and / or another type of computer-readable medium. Input component 510 may include a component that permits device 500 to receive information, such as via user input (e.g., a touch screen display, a keyboard, a keypad, a mouse, a button, a switch, a microphone, etc.). Additionally or alternatively, input component 510 may include a sensor for sensing information (e.g., a global positioning system (GPS) component, an accelerometer, a gyroscope, an actuator, etc.). Output component 512 may include a component that provides output information from device 500 (e.g., a display, a speaker, one or more light-emitting diodes (LEDs), etc.). Communication interface 514 may include a transceiver-like component (e.g., a transceiver, a separate receiver and transmitter, etc.) that enables device 500 to communicate with other devices, such as via a wired connection, a wireless connection, or a combination of wired and wireless connections. Communication interface 514 may permit device 500 to receiveinformation from another device and / or provide information to another device. For example, communication interface 514 may include an Ethernet interface, an optical interface, a coaxial interface, an infrared interface, a radio frequency (RF) interface, a universal serial bus (USB) interface, a Wi-Fi® interface, a cellular network interface, and / or the like.

[0115] Device 500 may perform one or more processes described herein. Device 500 may perform these processes based on processor 504 executing software instructions stored by a computer-readable medium, such as memory 506 and / or storage component 508. A computer-readable medium may include any non-transitory memory device. A memory device includes memory space located inside of a single physical storage device or memory space spread across multiple physical storage devices. Software instructions may be read into memory 506 and / or storage component 508 from another computer-readable medium or from another device via communication interface 514. When executed, software instructions stored in memory 506 and / or storage component 508 may cause processor 504 to perform one or more processes described herein. Additionally or alternatively, hardwired circuitry may be used in place of or in combination with software instructions to perform one or more processes described herein. Thus, embodiments described herein are not limited to any specific combination of hardware circuitry and software. The term “configured to,” as used herein, may refer to an arrangement of software, device(s), and / or hardware for performing and / or enabling one or more functions (e.g., actions, processes, steps of a process, and / or the like). For example, “a processor configured to” may refer to a processor that executes software instructions (e.g., program code) that cause the processor to perform one or more functions.

[0116] Although embodiments have been described in detail for the purpose of illustration, it is to be understood that such detail is solely for that purpose and that the disclosure is not limited to the disclosed embodiments or aspects, but, on the contrary, is intended to cover modifications and equivalent arrangements that are within the spirit and scope of the appended claims. For example, it is to be understood that the present disclosure contemplates that, to the extent possible, one or more features of any embodiment or aspect can be combined with one or more features of any other embodiment or aspect. In fact, any of these features can be combined in ways not specifically recited in the claims and / or disclosed in the specification. Although each dependent claim listed below may directly depend on only one claim, the disclosure ofpossible implementations includes each dependent claim in combination with every other claim in the claim set.

Claims

WHAT IS CLAIMED IS:1 . A computer-implemented method, comprising: receiving, with at least one processor, training data associated with a time series of a plurality of events; determining, with at least one processor, a plurality of time intervals between each pair of events of the plurality of events of the time series; generating, with at least one processor, a time interval embedding for each time interval of the plurality of time intervals to provide a plurality of time interval embeddings; calculating, with at least one processor, a plurality of attention weights for an attention mechanism of a machine learning model based on the plurality of time interval embeddings; and generating, with at least one processor, a trained machine learning model based on the attention weights for the attention mechanism of the machine learning model.

2. The computer-implemented method of claim 1 , wherein generating the time interval embedding for each time interval of the plurality of time intervals to provide the plurality of time interval embeddings comprises: determining an amount of time involved in each time interval of the plurality of time intervals; determining a discrete time interval of a plurality of discrete time intervals; and generating the time interval embedding for each time interval of the plurality of time intervals based on the discrete time interval associated with each time interval of the plurality of time intervals.

3. The computer-implemented method of claim 2, further comprising: discretizing each time interval of the plurality of time intervals based on the amount of time involved in each time interval to provide the plurality of discrete time intervals.

4. The computer-implemented method of claim 1 , further comprising: determining whether to perform an action based on an inference of the trained machine learning model.

5. The computer-implemented method of claim 1 , further comprising: calculating a plurality of attention scores of the machine learning model based on the plurality of time interval embeddings and the plurality of events, wherein each attention score comprises an output of the attention mechanism of the machine learning model based on an input provided to the attention mechanism; and wherein calculating the plurality of attention weights for the attention mechanism of the machine learning model comprises: calculating the plurality of attention weights for the attention mechanism of the machine learning model based on the plurality of attention scores of the machine learning model.

6. The computer-implemented method of claim 1 , wherein calculating the plurality of attention weights for the attention mechanism of the machine learning model comprises: executing a Tanh alignment function based on the plurality of time interval embeddings and the plurality of events to provide a time-embedded alignment factor; determining a first plurality of attention weights for the attention mechanism based on the plurality of time interval embeddings and the plurality of events; and multiplying the first plurality of attention weights for the attention mechanism by the time-embedded alignment factor to provide the plurality of attention weights for the attention mechanism of the machine learning model.

7. The computer-implemented method of claim 1 , wherein the machine learning model comprises a recurrent neural network (RNN) machine learning model.

8. A system for time series analysis using a time interval embedding based attention mechanism, comprising: at least one processor configured to: receive data associated with a time series of a plurality of events; determine a plurality of time intervals between each pair of events of the plurality of events of the time series; generate a time interval embedding for each time interval of the plurality of time intervals to provide a plurality of time interval embeddings; calculate a plurality of attention weights for an attention mechanism of a machine learning model based on the plurality of time interval embeddings; and generate a trained machine learning model based on the attention weights for the attention mechanism of the machine learning model.

9. The system of claim 8, wherein, when generating the time interval embedding for each time interval of the plurality of time intervals to provide the plurality of time interval embeddings, the at least one processor is configured to: determine an amount of time involved in each time interval of the plurality of time intervals; determine a discrete time interval of a plurality of discrete time intervals; and generate the time interval embedding for each time interval of the plurality of time intervals based on the discrete time interval associated with each time interval of the plurality of time intervals.

10. The system of claim 9, wherein the at least one processor is further configured to: discretize each time interval of the plurality of time intervals based on the amount of time involved in each time interval to provide the plurality of discrete time intervals.1 1 . The system of claim 8, wherein the at least one processor is further configured to:determine whether to perform an action based on an inference of the trained machine learning model.

12. The system of claim 8, wherein the at least one processor is further configured to: calculate a plurality of attention scores of the machine learning model based on the plurality of time interval embeddings and the plurality of events, wherein each attention score comprises an output of the attention mechanism of the machine learning model based on an input provided to the attention mechanism; and wherein, when calculating the plurality of attention weights for the attention mechanism of the machine learning model, the at least one processor is configured to: calculate the plurality of attention weights for the attention mechanism of the machine learning model based on the plurality of attention scores of the machine learning model.

13. The system of claim 8, wherein, when calculating the plurality of attention weights for the attention mechanism of the machine learning model, the at least one processor is configured to: execute a Tanh alignment function based on the plurality of time interval embeddings and the plurality of events to provide a time-embedded alignment factor; determine a first plurality of attention weights for the attention mechanism based on the plurality of time interval embeddings and the plurality of events; and multiply the first plurality of attention weights for the attention mechanism by the time-embedded alignment factor to provide the plurality of attention weights for the attention mechanism of the machine learning model.

14. The system of claim 8, wherein the machine learning model comprises a recurrent neural network (RNN) machine learning model.

15. A computer program product, including at least one non-transitory computer-readable medium including program instructions that, when executed by at least one processor, cause the at least one processor to:receive training data associated with a time series of a plurality of events; determine a plurality of time intervals between each pair of events of the plurality of events of the time series; generate a time interval embedding for each time interval of the plurality of time intervals to provide a plurality of time interval embeddings; calculate a plurality of attention weights for an attention mechanism of a machine learning model based on the plurality of time interval embeddings; and generate a trained machine learning model based on the attention weights for the attention mechanism of the machine learning model.

16. The computer program product of claim 15, wherein, the one or more program instructions that cause the at least one processor to generate the time interval embedding for each time interval of the plurality of time intervals to provide the plurality of time interval embeddings, cause the at least one processor to: determine an amount of time involved in each time interval of the plurality of time intervals; determine a discrete time interval of a plurality of discrete time intervals; and generate the time interval embedding for each time interval of the plurality of time intervals based on the discrete time interval associated with each time interval of the plurality of time intervals.

17. The computer program product of claim 16, wherein the one or more program instructions further cause the at least one processor to: discretize each time interval of the plurality of time intervals based on the amount of time involved in each time interval to provide the plurality of discrete time intervals.

18. The computer program product of claim 15, wherein the one or more program instructions further cause the at least one processor to: determine whether to perform an action based on an inference of the trained machine learning model.

19. The computer program product of claim 15, wherein the one or more program instructions further cause the at least one processor to: calculate a plurality of attention scores of the machine learning model based on the plurality of time interval embeddings and the plurality of events, wherein each attention score comprises an output of the attention mechanism of the machine learning model based on an input provided to the attention mechanism; and wherein, the one or more program instructions that cause the at least one processor to calculate the plurality of attention weights for the attention mechanism of the machine learning model, cause the at least one processor to: calculate the plurality of attention weights for the attention mechanism of the machine learning model based on the plurality of attention scores of the machine learning model.

20. The computer program product of claim 15, wherein, the one or more program instructions that cause the at least one processor to calculate the plurality of attention weights for the attention mechanism of the machine learning model, cause the at least one processor to: execute a Tanh alignment function based on the plurality of time interval embeddings and the plurality of events to provide a time-embedded alignment factor; determine a first plurality of attention weights for the attention mechanism based on the plurality of time interval embeddings and the plurality of events; and multiply the first plurality of attention weights for the attention mechanism by the time-embedded alignment factor to provide the plurality of attention weights for the attention mechanism of the machine learning model.21 . The computer program product of claim 15, wherein the machine learning model comprises a recurrent neural network (RNN) machine learning model.

Images