Monitoring system and method for the electronic monitoring of financial transactions

Abstract

The proposed solution relates in particular to an electronic monitoring system for the monitoring of financial transactions (T1, T2, T3, T4), the monitoring system (1) electronically checking financial transactions (T1, T2, T3, T4) between at least one payer (S1, S2, S3) and at least one payee (E) for the occurrence of at least one anomaly and the monitoring system (1) being configured to generate an alarm event upon at least one anomaly being detected. The monitoring system (1) comprises a tokenizer (10) and a language model (34) for the monitoring of the financial transactions (T1, T2, T3, T4), the tokenizer (10) being provided to generate a token sequence (w1-w5) from an input sequence (SQ) that is representative of a financial transaction (T1, T2, T3, T4), and the language model (34) being provided to check, on the basis of one token sequence (w1-w5) or a plurality of token sequences (w1-w5), the occurrence of an anomaly in one or more financial transactions (T1, T2, T3, T4) for the generation of an alarm event.

Description

[0001] Monitoring system and procedures for the electronic monitoring of financial transactions

[0002] Description

[0003] The proposed solution concerns, in particular, a monitoring system for the electronic surveillance of financial transactions.

[0004] The electronic monitoring of financial transactions increasingly uses machine learning tools to support, for example, financial institutions in the automated detection of anomalies in financial transactions. An anomaly is defined as a deviation from a predetermined set of rules that is indicative of a fraudulent financial transaction, in particular of at least one financial transaction used for money laundering. Electronic monitoring systems are designed to automatically filter out the portion of several thousand financial transactions per day that is classified as suspicious. An alarm event is then generated for a financial transaction classified as suspicious, enabling the respective financial transaction to be further investigated and, if necessary, reported to a supervisory and / or investigative authority.

[0005] Although elements of artificial intelligence (AI) are becoming increasingly important in this context, solutions for the efficient, cost-effective, and accurate detection of anomalies in financial transactions are hardly known. This is where the proposed solution comes in. In particular, a monitoring system for monitoring financial transactions is proposed that electronically checks financial transactions between at least one payer and at least one payee (or corresponding data on such financial transactions) for the occurrence of at least one anomaly. An anomaly is defined as a deviation from a specific set of rules that is indicative of a fraudulent financial transaction, in particular a financial transaction used for money laundering, and must therefore be classified as suspicious due to regulatory requirements.The proposed monitoring system is configured to generate an alarm event and thus an associated electronic alarm signal upon detection of at least one such anomaly, which triggers an alarm message, for example, on a (stationary or mobile) computer device of the monitoring system and / or on a computer device of a user system coupled to the monitoring system. The proposed monitoring system comprises a tokenizer and a language model for monitoring financial transactions. The tokenizer is provided to generate a token sequence from an input sequence representative of a financial transaction. The language model, in turn, is provided to check for the occurrence of an anomaly in one or more financial transactions based on one or more token sequences in order to generate an alarm event.

[0006] The proposed solution therefore uses a language model, which is not intended for processing natural language, but rather for processing tokenized input sequences for financial transactions. The proposed solution thus utilizes a language model based on a vocabulary for financial transaction data instead of a natural language vocabulary. This allows it to detect the extent to which anomalies exist in monitored financial transactions. These anomalies, for example, are represented by deviations in an input sequence evaluated for a financial transaction or a corresponding data set compared to a structure expected by the language model for the respective financial transaction.

[0007] An import sequence contains distinguishable transaction characteristics for each financial transaction. These transaction characteristics are treated like sentence words in a string passed to the tokenizer as an input sequence to generate input tokens for the language model. Tokenization of an input sequence is thus performed based on the transaction characteristics that were previously learned either automatically or manually. A transaction characteristic can be, for example, an account number, a bank code (Business Identifier Code, or BIC for short), a date, a time, or an indicator of a transaction type (e.g., transfer or withdrawal).

[0008] In one embodiment, the tokenizer is configured and provided to mask at least one input of the input tokens generated from the input sequence in a token sequence before the token sequence is passed to the language model. The monitoring system is then further configured to use the language model to predict an output token for the at least one masked input token. In this context, it can then be further provided, for example, that the monitoring system, in order to detect an anomaly, checks to what extent (in particular whether) the at least one predicted output token matches the associated input token. Thus, a prediction of the masked part of the partially masked token sequence of a financial transaction is made via the language model.If the language model cannot predict the masked input token(s), so that the output token does not match the original input token that was previously masked, the language model trained on the basis of permissible financial transactions concludes that a financial transaction or an associated money movement is suspicious and therefore an anomaly exists, which - possibly taking into account further criteria - leads to the triggering of an alarm event.

[0009] In this context, it can also be provided that a predefined, in particular percentage, portion of input tokens is always masked, and the monitoring system is configured to check, for anomaly detection, the extent to which predicted output tokens of an output sequence correspond to input tokens associated with the financial transaction (i.e., those that overmatch their position in the input sequence). If the language model cannot correctly predict the masked input tokens of one or more financial transactions, this indicates one or more suspicious financial transactions.

[0010] Alternatively or additionally, the monitoring system can be configured to detect an anomaly in a financial transaction by comparing the financial transactions using at least one reference input token of the token sequence associated with the financial transaction with at least one cluster formed on the basis of matching reference input tokens in token sequences for financial transactions. In this embodiment, the language model and the output tokens generated thereby are used to compare data relating to a financial transaction with one or more clusters, and to determine the extent to which the respective financial transaction can be classified as belonging to a cluster.This takes advantage of the recognition that financial transactions can be clustered using a language model if they have a specific, matching transaction characteristic and thus a resulting reference input token. However, if a financial transaction with an identical reference input token lies outside of such a cluster beyond a permissible limit, this indicates a suspicious financial transaction. For example, it can be observed that comparable financial transactions are always carried out with accounts of a certain merchant category. For example, a bakery's account typically has a large number of payment flows with comparatively small contributions from a variety of different private accounts during the bakery's opening hours.A financial transaction representing a payment flow with an amount many times larger than the average value outside of the bakery's opening hours would therefore lie outside a cluster formed for a bakery and would therefore be classified as suspicious - possibly taking other criteria into account.

[0011] With reference to the embodiment explained above, the reference input token used for clustering may be an input token of a token sequence that is representative of a merchant category code (MCC) contained in the input sequence.

[0012] For example, it is provided that the monitoring system is configured to determine a data point for comparison with the at least one cluster using the language model based on the at least one reference input token for the financial transaction and to determine its distance from a center of the at least one cluster. In such a procedure, for example, multidimensional, in particular high-dimensional output vectors from the language model are fed to a dimensionality reduction and then to a cluster function, i.e. a clustering algorithm, in order to form a data point in two-dimensional or three-dimensional space for an input sequence for a financial transaction, the distance of which data point from a center can be determined from different trained clusters. To detect an anomaly, this determined distance is compared with a stored threshold value.If the distance to a specific cluster or clusters is ultimately too large, a suspicious financial transaction is concluded and an alarm event is generated.

[0013] In a possible refinement, different thresholds can also be defined for different clusters. Thus, a greater variance of data points for financial transactions may be permissible for clusters with a specific first reference input token than for a cluster for financial transactions with a different, second reference input token.

[0014] In principle, both of the criteria explained above can also be considered for generating an alarm event. For example, an anomaly detection can be performed by checking, on the one hand, the extent to which (or whether) at least one predicted output token matches a corresponding masked input token, and, on the other hand, checking the extent to which a financial transaction can be assigned to a specific cluster of financial transactions based on an output token sequence. For example, if the language model cannot accurately predict the input tokens masked for a financial transaction and the language model also cannot reliably assign the financial transaction to a specific cluster, this indicates an anomaly and thus a suspicious financial transaction, which leads to an alarm event.If only one of the two criteria mentioned above indicates an anomaly, these can be weighted differently to decide whether the presence of one of the two criteria alone leads to the generation of an alarm event or not.

[0015] The language model-based monitoring system can, for example, be a transformer machine learning model (i.e., a transformer machine learning model). In particular, this can be a transformer machine learning model trained on input sequences of financial transactions, and in particular, a bidirectionally trained transformer machine learning model. For example, a transformer machine learning model trained bidirectionally with Bidirectional Encoder Representations from Transformers (BERT) or another bidirectional encoder transformer has proven particularly advantageous for reliably inferring anomalies in monitored financial transactions with comparatively manageable computational effort.

[0016] In principle, the alarm event can be provided for output on a computer device of the monitoring system and / or on a computer device of a user system connected to the monitoring system. The monitoring system can thus be implemented, for example, on the server side, in particular in the cloud. The generated alarm event is output on a local user system, for example, a financial institution. Alternatively, an on-premises monitoring system can be provided. In particular, a computer device of a user system can comprise, for example, a mobile device, such as a laptop or a mobile phone.

[0017] The proposed solution further relates to a method for electronically monitoring financial transactions between at least one payer and at least one payee for the occurrence of at least one anomaly. It is proposed to generate a token sequence from an input sequence representative of a financial transaction using a tokenizer and to check for the occurrence of an anomaly in one or more financial transactions using a language model based on one or more token sequences. An alarm event is generated in response to at least one anomaly detected during the check.

[0018] In particular, an embodiment variant of a proposed method can be implemented with an embodiment variant of a proposed monitoring system. Accordingly, the advantages and features of embodiment variants of a proposed monitoring system explained above and below also apply to embodiment variants of a proposed method, and vice versa.

[0019] In particular, in one embodiment of the proposed method, it can be provided that the tokenizer masks at least one input token of the input tokens generated from the input sequence in a token sequence before the token sequence is passed to the language model in order to then predict the at least one masked input token with the language model and to check an output token generated for this purpose from the language model to what extent it corresponds to the unmasked input token.

[0020] Alternatively or additionally, for detecting an anomaly, it may be provided that at least one reference input token for a financial transaction is used to check the extent to which this financial transaction can be assigned to a cluster of comparable financial transactions using the language model. Furthermore, the proposed solution relates to a computer program product comprising instructions that, when executed by at least one processor, cause the at least one processor to execute an embodiment of a proposed method.

[0021] Furthermore, the proposed solution also concerns a method for training a language model for monitoring financial transactions. The proposed solution therefore not only concerns the use of the already trained language model for detecting anomalies in financial transactions. Rather, it also focuses on how such a language model can be trained. In this context, it is proposed that such a training method include at least the following steps:

[0022] Providing data on individual financial transactions, typically in the form of a string,

[0023] Generating at least one set of input sequences for the financial transactions, each input sequence containing mutually distinguishable transaction characteristics (typically predefined and provided to the language model as input);

[0024] Tokenization of the input sequences based on the transaction characteristics using a tokenizer to generate input tokens for each input sequence;

[0025] Training the language model based on the input tokens and the input sequences by masking at least some of the input tokens to provide a language model trained on financial transaction data that can detect the occurrence of an anomaly in one or more financial transactions based on one or more token sequences.

[0026] The proposed method thus involves training the language model with the goal of ensuring that the language model correctly determines the masked input tokens from the provided context and thus checks the extent to which output tokens generated using the language model match input tokens that were previously provided to the language model only in masked form. According to this aspect of the proposed solution, a (possibly pre-trained) language model is used to learn a "vocabulary" based on tokenized input sequences for financial transactions.

[0027] In principle, in this context, it can be provided that the input tokens are each assigned to vectors of real numbers using word embedding. The tokenizer can first convert the transaction characteristics into numeric input tokens. If, from the perspective of the language model, a data set and thus a string that characterizes a financial transaction can be understood as a word, a set of input sequences for a specific account and / or a specific merchant category describes a text with characteristic words. As already explained above, the financial transactions underlying the at least one set of input sequences can thus, for example, be assigned to an account of a specific merchant category code when training the language model.A set of input sequences then forms, for example, a "text example" for a specific account type, which is identifiable by a reference input token (typically preceded by an input sequence). Separation or separation tokens can be provided between individual tokenized input sequences for the respective account.

[0028] For example, two input sequences for a particular account can be structured as follows:

[0029] (1 )

[0030] Trx1 Hp1 Cp019 2023-10-27:00:00:00 300EUR CASH DEBIT

[0031] ATM WITHDRAWAL DE MCC5892

[0032] (2)

[0033] Trx2 Hp1 Cp670 2023-10-30:01 :10:08 600EUR ACH CREDIT ACH TRANSFER US MCC5892

[0034] Using predefined transaction characteristics, these input sequences can be represented as follows:

[0035] (3)

[0036] 0D OH 0M OS AMOUNT2 CASH DEBIT ATM WITHDRAWAL DE

[0037] (4)

[0038] 3D 1 H 10M 8S AMOUNT3 ACH CREDIT ACH TRANSFER US

[0039] For example, the time ("2023-10-27:00:00:00") of the earliest financial transaction (1) is set as the zero point, and only the difference ("3D 1 H 10M 8S") is used for the date and time specification of the second financial transaction. Subsequently, the input sequences for the financial transactions (1) and (2) are converted into a numerical representation based on the formats (3) and (4) as follows:

[0040] (5)

[0041] 0 31 54 67 120 129 141 143 286

[0042] (6)

[0043] 3 32 65 74 121 132 140 148 232

[0044] Assuming that the corresponding account has a merchant category code MCC with the value “523”, then an input token sequence can be represented by the tokenizer for this account as follows:

[0045] [CLS] 523 [SEP] 3 32 65 74 121 132 140 148 232 [SEP] 0 31 54 67 120 129 141 143 286 [SEP] [PAD] [PAD] [PAD] [PAD] [PAD] [PAD] [PAD] [PAD] [PAD] [PAD] [PAD] [PAD] [PAD] [PAD] [PAD] [PAD] [PAD] [PAD] [PAD] [PAD] [PAD] [PAD] [PAD] [PAD] [PAD] [PAD] [PAD] [PAD] [PAD] [PAD] [PAD] [PAD] [PAD] [PAD] [PAD] [PAD] [PAD] [PAD] [PAD] [PAD] [PAD] [PAD] [PAD] [PAD] [PAD] [PAD] [PAD] [PAD] [PAD] [PAD]

[0046] Here, the token [CLF] is a common classification token. [SEP] is a separation token that separates two input sequences. [PAD] is a padding token used to reduce the number of input token sequences for an account to a uniform size across all training data.

[0047] Part of the tokenized transaction characteristics and thus the numeric input tokens can also be a categorization parameter for the transaction volume of the respective financial transaction. For example, a counterparty's IBAN is usually different for each account considered for which a financial transaction is being considered. Passing an IBAN as a transaction characteristic would make it difficult to transfer learned characteristics from one account to another, since each account and its considered transaction characteristics would result in a different set of IBANs and thus input tokens. Against this background, one implementation variant provides for categorization or sorting by transaction volume. For each account, the language model to be trained is provided with a set of input sequences containing a value for a categorization parameter.The values ​​for this categorization parameter depend on the transaction volume of the respective financial transactions with the counterparty's account. For example, a first value for this categorization parameter is always set identically for the account with the largest transaction volume. For example, in this context, it can be specified that an input token of "0" is always provided for the categorization parameter for the account with the largest transaction volume, and that, depending on the size of the transaction volumes occurring, further input tokens with ascending values ​​are set, e.g., an input token of "1" for the second-largest transaction volume, etc. This allows the language model to better generalize from one account to the next.

[0048] To train the language model, a variety of input sequences can be generated, based on accounts with identical and different merchant category codes. On this basis, output vectors generated with the language model as output—after optional dimensionality reduction and using a clustering function—can be used to create clusters that are representative of financial transactions of a merchant category code. These clusters can be used for later comparison to determine the extent to which a financial transaction to be examined with a specific merchant category code can be assigned to a corresponding cluster for this merchant category code using the trained language model, or whether it deviates so significantly from this that an anomaly can be assumed.

[0049] In principle, the language model to be trained can be a transformer machine learning model, in particular a transformer machine learning model bidirectionally pre-trained with BERT, a BERT-based encoder-transformer model, or another bidirectional encoder-transformer model.

[0050] The attached figures illustrate possible embodiments of the proposed solution

[0051] Here we show:

[0052] Figure 1 is a schematic representation of the monitoring system and its monitoring of financial transactions; Figure 2 is a set of input sequences for different

[0053] Financial transactions illustrating different transaction characteristics for a single transaction record;

[0054] Figure 3A shows the procedure for training a bidirectionally pre-trained

[0055] Transformer machine learning model based on input sequences according to Figure 2;

[0056] Figure 3A shows the procedure for checking financial transactions for possible

[0057] Anomalies based on the trained language model;

[0058] Figure 4A Visualization of different clusters for financial transactions identified with the trained language model;

[0059] Figure 4B shows a visualization for a language model-determined

[0060] Data point on a suspicious financial transaction, illustrating distances of this data point to different clusters;

[0061] Figure 5A is a schematic representation for assessing the relevance of

[0062] Financial transactions based on transaction characteristics or input tokens defined thereby;

[0063] Figure 5B shows a schematic representation for the processing of the relevance of different transaction characteristics of a financial transaction when generating an alarm event.

[0064] Figure 1 schematically shows an electronic monitoring system 1 with which financial transactions can be electronically monitored using a language model. Data on a large number of financial transactions, for example from one or more financial institutions, is made available to the monitoring system 1, or the monitoring system 1 is in use at a financial institution. Figure 1 schematically shows financial transactions T1 to T4 for the account of a payment recipient E, and therefore a beneficiary. With each financial transaction T1 to T4, the payment recipient E receives payment flows of varying amounts from different payment providers S1, S2, S3.Monitoring System 1 is used to check financial flows based on financial transactions T1 to T4 in order to electronically and automatically identify suspicious cases involving an anomaly in the respective financial transaction and thus, for example, a violation of money laundering regulations.

[0065] If an anomaly is detected in one of the financial transactions, monitoring system 1 automatically generates an alarm event. This alarm event then leads, for example, to the issuance of an alarm message to indicate a suspicious case in at least one of the financial transactions T1 to T4. Such an alarm message is then issued, for example, on a computer device of a user system 2 linked to monitoring system 1 in the course of anti-money laundering.

[0066] In one embodiment of the proposed solution, data on each financial transaction T1 to T4 is provided to a trained language model of the monitoring system 1 as an input sequence SQ with distinguishable transaction characteristics TC1 to TC5. An input sequence SQ according to Figure 2, for example, has a transaction text TT and thus a string composed of the multiple transaction characteristics TC1 to TC5.

[0067] A financial transaction T1 to T4 is uniquely identified by the sum of the transaction characteristics TC1 to TC5. For example, a first transaction characteristic TC1 contains information about the account of the payee E. A value for the transaction characteristic TC2, in turn, stands for the type of financial transaction and indicates, for example, whether it is a withdrawal from an ATM or a transfer. Another transaction characteristic TC3, for example, specifies the account number of the payer S1, S2 or S3. The transaction characteristic TC4, for example, identifies the bank sort code of the account of the payer S1, S2, S3. A transaction characteristic TC5, in turn, contains information about the amount of money. Further transaction characteristics, in turn, specify the date and time of the financial transactions.

[0068] To train a language model of the monitoring system 1, several input sequences SQ for an account can be provided in the form of words that serve as the basis for a vocabulary to be learned. For this purpose, a set of several financial transactions is randomly selected from financial transactions for a single account to provide a transaction data set TS and thus a set of input sequences SQ for training.

[0069] For both training the language model and its subsequent use, an input sequence SQ is tokenized. The individual transaction characteristics TC1 to TC5 are converted into numeric input tokens wi to W5.

[0070] For example, two input sequences for one account can be structured as follows:

[0071] (1 )

[0072] Trx1 Hp1 Cp019 2023-10-27:00:00:00 300EUR CASH DEBIT

[0073] ATM WITHDRAWAL DE MCC5892

[0074] (2)

[0075] Trx2 Hp1 Cp670 2023-10-30:01 :10:08 600EUR ACH CREDIT ACH TRANSFER US MCC5892

[0076] Using the predefined transaction characteristics TC1 to TCn, these input sequences can be represented as follows:

[0077] (3)

[0078] 0D OH 0M OS AMOUNT2 CASH DEBIT ATM WITHDRAWAL DE

[0079] (4)

[0080] 3D 1 H 10M 8S AMOUNT3 ACH CREDIT ACH TRANSFER US

[0081] For example, the time (“2023-10-27:00:00:00”) of the earliest financial transaction (1 ) is set as the zero point and only the difference (“3D 1 H 10M 8S”) is used for the date binding and time specification to the second financial transaction.

[0082] Subsequently, the input sequences for the financial transactions (1 ) and (2) are converted into a numerical representation for input tokens wi to w based on the formats (3) and (4). ntransferred as follows: (5)

[0083] 0 31 54 67 120 129 141 143 286

[0084] (6)

[0085] 3 32 65 74 121 132 140 148 232

[0086] Assuming that the corresponding account has a merchant category code MCC with the value “523”, then an input token sequence can be represented by the tokenizer for this account as follows:

[0087] [CLS] 523 [SEP] 3 32 65 74 121 132 140 148 232 [SEP] 0 31 54 67 120 129 141 143 286 [SEP] [PAD] [PAD] [PAD] [PAD] [PAD] [PAD] [PAD] [PAD] [PAD] [PAD] [PAD] [PAD] [PAD] [PAD] [PAD] [PAD] [PAD] [PAD] [PAD] [PAD] [PAD] [PAD] [PAD] [PAD] [PAD] [PAD] [PAD] [PAD] [PAD] [PAD] [PAD] [PAD] [PAD] [PAD] [PAD] [PAD] [PAD] [PAD] [PAD] [PAD] [PAD] [PAD] [PAD] [PAD] [PAD] [PAD] [PAD] [PAD] [PAD] [PAD]

[0088] Here, the token [CLF] represents a common classification token. [SEP] denotes a separation token that separates two input sequences. [PAD] denotes a padding token used to reduce a set of input token sequences (SQ) for an account to a uniform size for a transaction data set (TS).

[0089] Part of the tokenized transaction characteristics TC1 to TCn and thus the numerical input tokens can also be a categorization parameter for the transaction volume of the respective financial transaction T1 to T4. For example, the IBAN of a payee E is usually different for each account considered for which a financial transaction T1 to T4 is considered. If an IBAN were to be transferred as a transaction characteristic, this would complicate the transfer of learned characteristics from one account to another, since each account and its considered transaction characteristics TC1 to TCn would involve a different set of IBANs and thus input tokens w1 to w. nwould be fed in. Against this background, one embodiment variant provides for categorization or sorting according to transaction volume. Here, each account is assigned a value for a categorization parameter as part of a set of input sequences SQ, which in the example in Figure 1 are made available to the language model to be trained for a payer S1 to S3, thus an account of a counterparty. The values ​​for this categorization parameter depend on the transaction volume of the respective financial transactions T1 to T4 with the respective account (of the counterparty). A first value for this categorization parameter is always set identically, for example, for the account of a payer S1 to S3 with the largest transaction volume.For example, in this context, it can be provided that for the account with the largest transaction volume, an input token with the value "0" is always provided for the categorization parameter and, depending on the size of the transaction volumes occurring, further input tokens with ascending values ​​are set, e.g., an input token "1" for the second largest transaction volume, etc. This allows the language model to better generalize from one account to the next.

[0090] The training of the language model used in the monitoring system 1 for monitoring financial transactions T1 to T4 is illustrated in more detail in Figure 3A. A language model 31 with a bidirectional transformer encoder is provided. Input sequences SQ, or a set of input sequences SQ, are fed to a tokenizer 10. This tokenizer 10 generates a token sequence with the input tokens w1 to W5 for the financial transactions based on the transaction characteristics TC1 to TC5 of each input sequence SQ.

[0091] To train the language model 31, a certain portion of the input tokens w1 to w5 is masked, exemplified in the example in Figure 3 by replacing the input token W4 with a masked token [MASK]. The token sequence is then fed to a word embedding 30 to prepare the token sequence for the language model 31. This preparation includes the above-explained addition of the classification token [CLS], the separation tokens [SEP], and the filler tokens [PAD].

[0092] A token sequence then passes through a transformer encoder layer of the language model 31. For a partially masked token sequence, the language model 31 generates multidimensional output vectors 01 to 05 in a manner known for BERT, from which output tokens w'i to w'5 are derived after classification 32 and subsequent vocabularyization 33. The output vectors 01 to 05 correspond accordingly to the respective input tokens wi to w5. The output vectors 01 to 05 therefore contain the context information of the respective input token wi to W5, so that it is possible to learn in which context a certain transaction characteristic occurs. During classification 32, one or more classification layers are passed through, which generate probabilities for different classes in a classification task, so that output tokens for masked input tokens, such as e.g.B. an output token w'4 is inferred for the masked input token W4. The language model 31 is therefore trained with a large number of sets of input sequences SQ with the goal of inferring masked input tokens such as the input token W4 based on the context, i.e., unmasked input tokens w1, W2, W3, W5. For example, 15% of the input tokens w1 to W5 can be randomly selected for masking. Of these, i) 80% are replaced by a masked token [MASK] according to Figure 3A, ii) 10% are replaced with a random input token, and iii) 10% are left unchanged. By comparing the generated output tokens w'i to w'5 with the token sequence of the input tokens w1 to W5, the language model 31 is trained to correctly predict input tokens and thus infer transaction characteristics TC1 to TC5.

[0093] Once the language model has been trained with appropriate training data, a trained language model 34 as shown in Figure 3B is available for the efficient monitoring of financial transactions. For each financial transaction T1 to T4, an input sequence SQ is made available to the tokenizer 10 individually or as part of a set of multiple input sequences SQ for an account. A specific portion of the input tokens w1 to W5 is also masked here. As the trained language model 34 passes through the transformer encoder level, high-dimensional output vectors W1 to W5 are generated, which are used to predict output vectors W4 for masked input tokens. If predicted output tokens, and thus an output token sequence, deviate from the actual token sequence W1-W5, this indicates an anomaly in the respective financial transaction. That is,In other words, it checks the extent to which previously masked input tokens for transaction characteristics are correctly predicted based on the trained language model 34. If this is not the case, the transaction characteristics TC1 to TC5 of a financial transaction deviate noticeably from the normal case. This makes the respective financial transaction appear suspicious and triggers an alarm event for further investigation by a user or a control authority, for example, at a financial institution. A corresponding anomaly detection 35 is provided in Figure 3B.

[0094] The anomaly detection 35 may, if appropriate, additionally include that after a dimensional reduction of the high-dimensional output vectors 01 to 05 for a financial transaction, a data point is determined and compared with clusters generated during the training of the language model 31 for the financial transactions considered so far.

[0095] Thus, according to the visualization in Figure 4, it can be shown that financial transactions classified as a reference input token for an account with a specific merchant category code MCC can be assigned to 31 different clusters C1 to C9 specific to merchant category codes using the language model. However, if a data point A for a financial transaction according to Figure 4B is not located near one of these clusters C1 to C9, this also indicates an anomaly in the financial transaction belonging to data point A. For example, in the visualized example in Figure 4B, data point A is spaced at a distance d1 from a center M1 of one cluster C8 and at a distance d2 from a center M2 of another cluster C5, such that distance d1 is above a threshold specified for cluster C8 and distance d2 is above a threshold specified for cluster C5.

[0096] If, for example, within the scope of the anomaly detection 35 of Figure 3B, it is determined for a financial transaction using the trained language model 34 that the language model 34 does not fully accurately predict the token sequence wi to W5 and, in addition, a data point A determined for this purpose cannot be assigned to a previously determined cluster C5 or C8 for similar financial transactions, this indicates an anomaly and thus a suspicious financial transaction that triggers an alarm event.

[0097] In order to make a correspondingly triggered alarm event easier to evaluate during the evaluation of a further check by a human user, the embodiment variant of Figure 3B provides a relevance assessment 36 after the anomaly detection 35. Here, the criteria that were decisive for the language model-based assessment of a financial transaction as suspicious are prepared and comprehensibly visualized for a human user when the alarm event is issued.

[0098] For example, each financial transaction T1 to T4 can be represented as the sum of transaction characteristics or corresponding transaction tokens 500, 501, which are assigned to different groups 50 and 51. For example, the transaction tokens 500 together represent a payment type 50, and the transaction tokens 501 represent a country code 51. For example, if one of the transaction tokens 500 for the payment type 50 in the financial transaction T1 is classified as highly relevant with regard to the detection of an anomaly, the financial transaction is assigned an importance score of 1. In contrast, the financial transaction T3 contains a relevance score of 2, while the two other pieces of information displayed, T2 and T4, are assigned a relevance score of 3.In a visualization of the various financial transactions T1 to T4 on a user system 2, financial transaction T1 is highlighted as potentially more relevant for a closer examination of the occurrence of an anomaly. The trained language model 34 thus enables a token-based explanation for classifying various financial transactions as potentially suspicious or non-suspicious.

[0099] This principle also underlies a visualization according to Figure 5B. While for the visualization within the context of the relevance assessment 36 according to Figure 5A, token-based differences across multiple financial transactions T1 to T4 are visualized, a visualization according to the procedure in Figure 5B is intended to visualize the different transaction characteristics within a financial transaction T1 for a user of the user system 2 that have led to an assessment of this financial transaction T1 as suspicious and thus as the detection of an anomaly. For example, if it can be concluded based on the transaction tokens 500, 501 and thus the output tokens of the trained language model 34 predicted for the corresponding transaction characteristics that there are significant deviations from financial transactions that are to be classified as permissible, particularly with regard to the payment type and country code, these deviations can be weighted and visualized accordingly.

[0100] For example, after several similarly conspicuous financial transactions have occurred, a user can be informed that, in the case of the alarm event triggered for an account, in particular on the basis of the financial transaction T1, the decisive factor in each case was that small amounts were repeatedly transferred in quick succession to a specific account of a payment recipient E, that a large part of the transaction volume was part of a round-trip transaction and that it was at least partly attributable to a cross-border financial transaction.

[0101] A token-based visualization can significantly improve explainability for a user of the monitoring system 1. This offers an additional advantage for the improved technology associated with the proposed solution for efficient electronic, automated monitoring of financial transactions based on a language model, which can be implemented with comparatively low processor power and high speed.

[0102] List of reference symbols

[0103] 1 surveillance system

[0104] 10 Tokenizer

[0105] 2 user system

[0106] 30 word embedding

[0107] 31 Pre-trained language model / transformation encoder

[0108] 32 Classification

[0109] 33 Vocabulary

[0110] 34 Trained language model

[0111] 35 Anomaly detection and clustering

[0112] 36 Relevance assessment

[0113] 50 Payment type

[0114] 500 transaction tokens for payment type

[0115] 501 Country Code Transaction Token

[0116] 51 Country code

[0117] A data point for anomaly

[0118] C1 - C9 cluster d1 , d2 distance

[0119] E Payment recipient

[0120] I Relevance value

[0121] M1, M2 cluster center

[0122] 01, 02, 03, 04, 05 output vector

[0123] S1, S2, S3 Payer

[0124] SQ input sequence

[0125] T1, T2, T3, T4 financial transaction

[0126] TC1 - TC5 T ransaction characteristic

[0127] TS transaction record (set of input sequences)

[0128] TT transaction text w'i,w'2, w'3, w'4, w'5 output token

[0129] Wi , W2, W3, W4, W5 input tokens

Claims

Claims 1 . Monitoring system for monitoring financial transactions (T1, T2, T3, T4), wherein the monitoring system (1) electronically checks financial transactions (T1, T2, T3, T4) between at least one payment provider (S1, S2, S3) and at least one payment recipient (E) for the occurrence of at least one anomaly, and the monitoring system (1) is configured to generate an alarm event upon detection of at least one anomaly, characterized in that the monitoring system (1) comprises a tokenizer (10) and a language model (34) for monitoring the financial transactions (T1, T2, T3, T4), wherein the tokenizer (10) is provided to generate a token sequence (wi-w5) from an input sequence (SQ) representative of a financial transaction (T1, T2, T3, T4), and the language model (34) is provided to generate, on the basis of one token sequence (wi-w5) or several token sequences (W1-W5) the occurrence of an anomaly in one or more financial transactions (T 1 , T2, T3,T4) for generating an alarm event.

2. Monitoring system according to claim 1, characterized in that the tokenizer (10) is set up and provided to mask at least one input token (w4) of the input tokens (w1, W2, W3, w4, W5) generated from the input sequence (SQ) in a token sequence (W1-W5) before the token sequence (W1-W5) is transferred to the language model (34), and the monitoring system is configured to use the language model (34) to predict an output token (w'4) for the at least one masked input token (w4).

3. Monitoring system according to claim 2, characterized in that the monitoring system is configured to check, for the detection of an anomaly, the extent to which the at least one predicted output token (w'4) corresponds to the associated input token (w4).

4. Monitoring system according to claim 3, characterized in that the tokenizer (10) is designed and provided to mask a predetermined portion of the input tokens (w1, W2, W3, w4, W5) in a token sequence (W1-W5), and the monitoring system is configured to check for the detection of an anomaly, to what extent predicted output tokens (w'i,w'2, w'3, w'4, w'5) match with corresponding input tokens (wi,W2, W3, W4, W5).

5. Monitoring system according to one of the preceding claims, characterized in that the monitoring system (1) is configured to compare, for the detection of an anomaly, a financial transaction (T1, T2, T3, T4) on the basis of at least one reference input token (wi) of the token sequence (W1-W5) associated with the financial transaction (T1, T2, T3, T4) with at least one cluster (C1-C9) which is formed on the basis of token sequences for financial transactions (T1, T2, T) with mutually matching reference input tokens (wi).

6. Monitoring system according to claim 5, characterized in that the reference input token (wi) is in each case an input token of a token sequence (W1-W5) which is representative of a dealer category code contained in the input sequence (SQ).

7. Monitoring system according to claim 5 or 6, characterized in that the monitoring system (1) is configured to check, for the detection of an anomaly, to what extent the respective financial transaction (T1, T2, T3, T4) can be assigned to the at least one cluster (C1-C9).

8. Monitoring system according to one of claims 5 to 7, characterized in that the monitoring system (1) is configured to determine a data point (A) for the comparison with the at least one cluster (C1 -C9) with the language model (34) on the basis of the at least one reference input token (wi) for the financial transaction (T1, T2, T3, T4) and to determine its distance (d1, d2) to a center (M1, M2) of the at least one cluster (C1 -C9).

9. Monitoring system according to claim 8, characterized in that the monitoring system (1) is configured to compare the determined distance (d1, d2) with a stored threshold value for the detection of an anomaly.

10. Monitoring system according to claim 9, characterized in that different threshold values ​​are provided for different clusters (C1 -C9).

11. Monitoring system according to one of claims 8 to 10, characterized in that the monitoring system (1) is configured to determine the data point (A) by performing a dimensional reduction for multidimensional output vectors (O1-O5) obtained from the language model (34) for input tokens (wi, w2, w3, w4, w5) of a financial transaction (T1, T2, T3, T4).

12. Monitoring system according to one of claims 3 or 4 and one of claims 5 to 11, characterized in that the monitoring system (1) is configured, for the detection of an anomaly, on the one hand to check the extent to which the at least one predicted output token (w'4) matches the associated input token (w4) and on the other hand to carry out a comparison with the at least one cluster (C1 -C9) on the basis of the at least one reference input token (wi).

13. Monitoring system according to one of the preceding claims, characterized in that the language model (34) of the monitoring system (1) is a transformer machine learning model, in particular a transformer machine learning model trained on the basis of input sequences (SQ) for financial transactions (T1, T2, T3, T4) and in particular a transformer machine learning model trained bidirectionally on the basis of input sequences (SQ) for financial transactions (T1, T2, T3, T4).

14. Monitoring system according to one of the preceding claims, characterized in that the alarm event is provided for output to a computer device of the monitoring system (1) and / or to a computer device of a user system (2) connected to the monitoring system (1).

15. Method for electronically monitoring financial transactions (T1, T2, T3, T4) between at least one payer (S1, S2, S3) and at least one payee (E) for the occurrence of at least one anomaly, wherein an alarm event is generated in response to at least one anomaly detected during the check, characterized in that a token sequence (W1-W5) is generated from an input sequence (SQ) representative of a financial transaction (T1, T2, T2) via a tokenizer (10) and the occurrence of an anomaly in one or more financial transactions (T1, T2, T) is checked for the generation of an alarm event using a language model (34) based on a token sequence (W1-W5) or a plurality of token sequences (W1-W5).

16. A computer program product comprising instructions which, when executed by at least one processor, cause the at least one processor to execute the method according to claim 15.

17. Method for training a language model (31 ) for monitoring Financial transactions comprising at least the following steps: - Providing data (TT) on individual financial transactions (T1, T2, T3, T4); - generating at least one set of input sequences (SQ) for the financial transactions, each input sequence (SQ) containing mutually distinguishable transaction characteristics (TC1 -TC5); - Tokenization of the input sequences (SQ) based on the transaction characteristics (TC1 -TC5) with a tokenizer (10) to generate input tokens (W1-W5) for each input sequence (SQ); - Training the language model (31) on the basis of the input tokens (wi, w2, w3, w4, w5) and the input sequences (SQ) by masking at least some of the input tokens (wi, w2, W3, W4, W5) in order to provide a language model (34) trained on data relating to financial transactions (T1, T2, T2), with which the occurrence of an anomaly in one or more financial transactions (T1, T2, T) can be detected on the basis of one token sequence (W1-W5) or several token sequences (W1-W5).

18. The method according to claim 17, characterized in that the input tokens (wi,w2, W3, w4, W5) are assigned to vectors of real numbers by means of word embedding.

19. The method according to claim 17 or 18, characterized in that the tokenizer (10) converts the transaction characteristics (TC1 -TC5) into numerical input tokens (wi, w2, w3, w4, W5).

20. The method according to claim 19, characterized in that the financial transactions (T1, T2, T3, T4) underlying the at least one set of input sequences (SQ) are assigned to an account of a specific merchant category code.

21. Method according to one of claims 17 to 20, characterized in that a plurality of sets of input sequences (SQ) are generated, these sets of input sequences (SQ) being based on accounts with different merchant category codes, and on the basis of multidimensional output vectors (O1-O5) generated as output by the language model (31), after a dimensional reduction, clusters (C1-C9) are generated via a cluster function, which are each representative of financial transactions (T1, T2, T3, T4) of a merchant category code.

22. Method according to one of claims 17 to 21, characterized in that at least one of the tokenized transaction characteristics (TC1 -TC5) relates to a categorization parameter for a transaction volume of the respective financial transaction (T1, T2, T3, T4).

23. Method according to claim 22, characterized in that values ​​for the Categorisation parameters are specified in ascending or descending order, starting with a value for the counterparty account (S1 -S3) that has the largest transaction volume in view of the data provided (TT) on the financial transactions (T1, T2, T3, T4) with an account.

24. Method according to one of claims 17 to 23, characterized in that the language model (31) to be trained is a transformer machine learning model, in particular a bi-directionally pre-trained transformer machine learning model.

25. Use of a monitoring system according to one of claims 1 to 14 for Monitoring of financial transactions.

Images